Uses of Class
opennlp.tools.tokenize.TokenSample
-
Packages that use TokenSample Package Description opennlp.tools.cmdline.tokenizer opennlp.tools.formats Experimental package related to converting various corpora to OpenNLP Format.opennlp.tools.formats.ad Experimental package related to theArvores Deitadas corpusformat.opennlp.tools.formats.conllu Experimental package related to the CoNNL-U format.opennlp.tools.formats.convert opennlp.tools.formats.irishsentencebank Experimental package related to the Irish Sentence Bank format.opennlp.tools.formats.masc Experimental package related to theMASCcorpus format.opennlp.tools.tokenize Contains classes related to finding token or words in a string.opennlp.tools.tokenize.lang.en -
-
Uses of TokenSample in opennlp.tools.cmdline.tokenizer
Methods in opennlp.tools.cmdline.tokenizer with parameters of type TokenSample Modifier and Type Method Description voidDetokenEvaluationErrorListener. misclassified(TokenSample reference, TokenSample prediction)voidTokenEvaluationErrorListener. misclassified(TokenSample reference, TokenSample prediction) -
Uses of TokenSample in opennlp.tools.formats
Methods in opennlp.tools.formats that return types with arguments of type TokenSample Modifier and Type Method Description ObjectStream<TokenSample>ConllXTokenSampleStreamFactory. create(String[] args)ObjectStream<TokenSample>TokenSampleStreamFactory. create(String[] args) -
Uses of TokenSample in opennlp.tools.formats.ad
Methods in opennlp.tools.formats.ad that return types with arguments of type TokenSample Modifier and Type Method Description ObjectStream<TokenSample>ADTokenSampleStreamFactory. create(String[] args) -
Uses of TokenSample in opennlp.tools.formats.conllu
Methods in opennlp.tools.formats.conllu that return TokenSample Modifier and Type Method Description TokenSampleConlluTokenSampleStream. read()Methods in opennlp.tools.formats.conllu that return types with arguments of type TokenSample Modifier and Type Method Description ObjectStream<TokenSample>ConlluTokenSampleStreamFactory. create(String[] args) -
Uses of TokenSample in opennlp.tools.formats.convert
Methods in opennlp.tools.formats.convert that return TokenSample Modifier and Type Method Description TokenSampleNameToTokenSampleStream. read()TokenSamplePOSToTokenSampleStream. read()Methods in opennlp.tools.formats.convert that return types with arguments of type TokenSample Modifier and Type Method Description ObjectStream<TokenSample>NameToTokenSampleStreamFactory. create(String[] args)ObjectStream<TokenSample>ParseToTokenSampleStreamFactory. create(String[] args)ObjectStream<TokenSample>POSToTokenSampleStreamFactory. create(String[] args) -
Uses of TokenSample in opennlp.tools.formats.irishsentencebank
Methods in opennlp.tools.formats.irishsentencebank that return TokenSample Modifier and Type Method Description TokenSampleIrishSentenceBankDocument.IrishSentenceBankSentence. getTokenSample()Methods in opennlp.tools.formats.irishsentencebank that return types with arguments of type TokenSample Modifier and Type Method Description ObjectStream<TokenSample>IrishSentenceBankTokenSampleStreamFactory. create(String[] args) -
Uses of TokenSample in opennlp.tools.formats.masc
Methods in opennlp.tools.formats.masc that return TokenSample Modifier and Type Method Description TokenSampleMascTokenSampleStream. read()Methods in opennlp.tools.formats.masc that return types with arguments of type TokenSample Modifier and Type Method Description ObjectStream<TokenSample>MascTokenSampleStreamFactory. create(String[] args) -
Uses of TokenSample in opennlp.tools.tokenize
Methods in opennlp.tools.tokenize that return TokenSample Modifier and Type Method Description static TokenSampleTokenSample. parse(String sampleString, String separatorChars)Parses a string sample.protected TokenSampleDetokenizerEvaluator. processSample(TokenSample reference)protected TokenSampleTokenizerEvaluator. processSample(TokenSample reference)TokenSampleTokenizerStream. read()TokenSampleTokenSampleStream. read()Methods in opennlp.tools.tokenize with parameters of type TokenSample Modifier and Type Method Description protected Iterator<Event>TokSpanEventStream. createEvents(TokenSample tokenSample)Adds training events to the event stream for each of the specifiedsample.protected TokenSampleDetokenizerEvaluator. processSample(TokenSample reference)protected TokenSampleTokenizerEvaluator. processSample(TokenSample reference)Method parameters in opennlp.tools.tokenize with type arguments of type TokenSample Modifier and Type Method Description voidTokenizerCrossValidator. evaluate(ObjectStream<TokenSample> samples, int nFolds)Starts the evaluation.static TokenizerModelTokenizerME. train(ObjectStream<TokenSample> samples, TokenizerFactory factory, TrainingParameters mlParams)Trains a model for theTokenizerME.Constructor parameters in opennlp.tools.tokenize with type arguments of type TokenSample Constructor Description TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics)Initializes a new event stream based on the data stream using aTokenContextGeneratorthat relies on aDefaultTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg)Initializes a new event stream based on the data stream using aTokenContextGenerator.TokSpanEventStream(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg)Initializes a new event stream based on the data stream using aTokenContextGenerator.WhitespaceTokenStream(ObjectStream<TokenSample> tokens)Initializes aWhitespaceTokenStream. -
Uses of TokenSample in opennlp.tools.tokenize.lang.en
Methods in opennlp.tools.tokenize.lang.en that return TokenSample Modifier and Type Method Description TokenSampleTokenSampleStream. next()
-