| Package | Description |
|---|---|
| net.sf.okapi.steps.tokenization.engine.javacc |
| Modifier and Type | Class and Description |
|---|---|
class |
WordTokenizer
A grammar-based tokenizer constructed with JavaCC.
|
class |
WordTokenizerTokenManager
Token Manager.
|
Copyright © 2018. All rights reserved.