|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||
| Packages that use TokenSample | |
|---|---|
| opennlp.tools.cmdline.tokenizer | |
| opennlp.tools.formats | Experimental package related to converting various corpora to OpenNLP Format. |
| opennlp.tools.formats.ad | |
| opennlp.tools.formats.convert | |
| opennlp.tools.tokenize | Contains classes related to finding token or words in a string. |
| opennlp.tools.tokenize.lang.en | |
| Uses of TokenSample in opennlp.tools.cmdline.tokenizer |
|---|
| Methods in opennlp.tools.cmdline.tokenizer with parameters of type TokenSample | |
|---|---|
void |
TokenEvaluationErrorListener.missclassified(TokenSample reference,
TokenSample prediction)
|
| Uses of TokenSample in opennlp.tools.formats |
|---|
| Methods in opennlp.tools.formats that return types with arguments of type TokenSample | |
|---|---|
ObjectStream<TokenSample> |
TokenSampleStreamFactory.create(String[] args)
|
ObjectStream<TokenSample> |
ConllXTokenSampleStreamFactory.create(String[] args)
|
| Uses of TokenSample in opennlp.tools.formats.ad |
|---|
| Methods in opennlp.tools.formats.ad that return types with arguments of type TokenSample | |
|---|---|
ObjectStream<TokenSample> |
ADTokenSampleStreamFactory.create(String[] args)
|
| Uses of TokenSample in opennlp.tools.formats.convert |
|---|
| Methods in opennlp.tools.formats.convert that return TokenSample | |
|---|---|
TokenSample |
POSToTokenSampleStream.read()
|
TokenSample |
NameToTokenSampleStream.read()
|
| Methods in opennlp.tools.formats.convert that return types with arguments of type TokenSample | |
|---|---|
ObjectStream<TokenSample> |
POSToTokenSampleStreamFactory.create(String[] args)
|
ObjectStream<TokenSample> |
NameToTokenSampleStreamFactory.create(String[] args)
|
ObjectStream<TokenSample> |
ParseToTokenSampleStreamFactory.create(String[] args)
|
| Uses of TokenSample in opennlp.tools.tokenize |
|---|
| Methods in opennlp.tools.tokenize that return TokenSample | |
|---|---|
static TokenSample |
TokenSample.parse(String sampleString,
String separatorChars)
|
protected TokenSample |
TokenizerEvaluator.processSample(TokenSample reference)
|
TokenSample |
TokenSampleStream.read()
|
TokenSample |
TokenizerStream.read()
|
| Methods in opennlp.tools.tokenize with parameters of type TokenSample | |
|---|---|
protected Iterator<opennlp.model.Event> |
TokSpanEventStream.createEvents(TokenSample tokenSample)
Adds training events to the event stream for each of the specified tokens. |
protected TokenSample |
TokenizerEvaluator.processSample(TokenSample reference)
|
| Method parameters in opennlp.tools.tokenize with type arguments of type TokenSample | |
|---|---|
void |
TokenizerCrossValidator.evaluate(ObjectStream<TokenSample> samples,
int nFolds)
Starts the evaluation. |
static TokenizerModel |
TokenizerME.train(ObjectStream<TokenSample> samples,
TokenizerFactory factory,
TrainingParameters mlParams)
Trains a model for the TokenizerME. |
static TokenizerModel |
TokenizerME.train(String languageCode,
ObjectStream<TokenSample> samples,
boolean useAlphaNumericOptimization)
Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters)
and pass in a TokenizerFactory |
static TokenizerModel |
TokenizerME.train(String languageCode,
ObjectStream<TokenSample> samples,
boolean useAlphaNumericOptimization,
int cutoff,
int iterations)
Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters)
and pass in a TokenizerFactory |
static TokenizerModel |
TokenizerME.train(String languageCode,
ObjectStream<TokenSample> samples,
boolean useAlphaNumericOptimization,
TrainingParameters mlParams)
Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters)
and pass in a TokenizerFactory |
static TokenizerModel |
TokenizerME.train(String languageCode,
ObjectStream<TokenSample> samples,
Dictionary abbreviations,
boolean useAlphaNumericOptimization,
TrainingParameters mlParams)
Deprecated. Use #train(String, ObjectStream, TokenizerFactory, TrainingParameters)
and pass in a TokenizerFactory |
| Constructor parameters in opennlp.tools.tokenize with type arguments of type TokenSample | |
|---|---|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics)
Initializes the current instance. |
|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics,
Pattern alphaNumeric,
TokenContextGenerator cg)
Initializes the current instance. |
|
TokSpanEventStream(ObjectStream<TokenSample> tokenSamples,
boolean skipAlphaNumerics,
TokenContextGenerator cg)
Initializes the current instance. |
|
WhitespaceTokenStream(ObjectStream<TokenSample> tokens)
|
|
| Uses of TokenSample in opennlp.tools.tokenize.lang.en |
|---|
| Methods in opennlp.tools.tokenize.lang.en that return TokenSample | |
|---|---|
TokenSample |
TokenSampleStream.next()
|
|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||