| Package | Description |
|---|---|
| net.sf.okapi.steps.tokenization.engine.javacc |
| Modifier and Type | Field and Description |
|---|---|
protected SimpleCharStream |
WordTokenizerTokenManager.input_stream |
| Modifier and Type | Method and Description |
|---|---|
void |
WordTokenizerTokenManager.ReInit(SimpleCharStream stream)
Reinitialise parser.
|
void |
WordTokenizerTokenManager.ReInit(SimpleCharStream stream,
int lexState)
Reinitialise parser.
|
| Constructor and Description |
|---|
WordTokenizerTokenManager(SimpleCharStream stream)
Constructor.
|
WordTokenizerTokenManager(SimpleCharStream stream,
int lexState)
Constructor.
|
Copyright © 2021. All rights reserved.