| Package | Description |
|---|---|
| opennlp.tools.tokenize |
Contains classes related to finding token or words in a string.
|
| opennlp.tools.tokenize.lang |
| Modifier and Type | Class and Description |
|---|---|
class |
DefaultTokenContextGenerator
Generate events for maxent decisions for tokenization.
|
| Modifier and Type | Method and Description |
|---|---|
TokenContextGenerator |
TokenizerFactory.getContextGenerator()
Gets the context generator
|
| Constructor and Description |
|---|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics,
Pattern alphaNumeric,
TokenContextGenerator cg)
Initializes the current instance.
|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics,
TokenContextGenerator cg)
Initializes the current instance.
|
| Modifier and Type | Method and Description |
|---|---|
TokenContextGenerator |
Factory.createTokenContextGenerator(String languageCode,
Set<String> abbreviations) |
Copyright © 2017 The Apache Software Foundation. All rights reserved.