|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||
| Packages that use TokenSample | |
|---|---|
| opennlp.tools.cmdline.tokenizer | |
| opennlp.tools.formats | Experimental package related to converting various corpora to OpenNLP Format. |
| opennlp.tools.tokenize | Contains classes related to finding token or words in a string. |
| opennlp.tools.tokenize.lang.en | |
| Uses of TokenSample in opennlp.tools.cmdline.tokenizer |
|---|
| Methods in opennlp.tools.cmdline.tokenizer that return types with arguments of type TokenSample | |
|---|---|
protected ObjectStreamFactory<TokenSample> |
TokenizerConverterTool.createStreamFactory(java.lang.String format)
|
| Uses of TokenSample in opennlp.tools.formats |
|---|
| Methods in opennlp.tools.formats that return TokenSample | |
|---|---|
TokenSample |
POSToTokenSampleStream.read()
|
TokenSample |
NameToTokenSampleStream.read()
|
| Methods in opennlp.tools.formats that return types with arguments of type TokenSample | |
|---|---|
ObjectStream<TokenSample> |
POSToTokenSampleStreamFactory.create(java.lang.String[] args)
|
ObjectStream<TokenSample> |
NameToTokenSampleStreamFactory.create(java.lang.String[] args)
|
ObjectStream<TokenSample> |
ConllXTokenSampleStreamFactory.create(java.lang.String[] args)
|
| Uses of TokenSample in opennlp.tools.tokenize |
|---|
| Methods in opennlp.tools.tokenize that return TokenSample | |
|---|---|
static TokenSample |
TokenSample.parse(java.lang.String sampleString,
java.lang.String separatorChars)
|
TokenSample |
TokenSampleStream.read()
|
TokenSample |
TokenizerStream.read()
|
| Methods in opennlp.tools.tokenize with parameters of type TokenSample | |
|---|---|
protected java.util.Iterator<Event> |
TokSpanEventStream.createEvents(TokenSample tokenSample)
Adds training events to the event stream for each of the specified tokens. |
void |
TokenizerEvaluator.evaluateSample(TokenSample reference)
Evaluates the given reference TokenSample object. |
| Method parameters in opennlp.tools.tokenize with type arguments of type TokenSample | |
|---|---|
void |
TokenizerCrossValidator.evaluate(ObjectStream<TokenSample> samples,
int nFolds)
|
static TokenizerModel |
TokenizerME.train(java.lang.String languageCode,
ObjectStream<TokenSample> samples,
boolean useAlphaNumericOptimization)
Trains a model for the TokenizerME with a default cutoff of 5 and 100 iterations. |
static TokenizerModel |
TokenizerME.train(java.lang.String languageCode,
ObjectStream<TokenSample> samples,
boolean useAlphaNumericOptimization,
int cutoff,
int iterations)
Trains a model for the TokenizerME. |
| Constructor parameters in opennlp.tools.tokenize with type arguments of type TokenSample | |
|---|---|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics)
Initializes the current instance. |
|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics,
TokenContextGenerator cg)
Initializes the current instance. |
|
WhitespaceTokenStream(ObjectStream<TokenSample> tokens)
|
|
| Uses of TokenSample in opennlp.tools.tokenize.lang.en |
|---|
| Methods in opennlp.tools.tokenize.lang.en that return TokenSample | |
|---|---|
TokenSample |
TokenSampleStream.next()
|
|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||