Uses of Package
opennlp.tools.tokenize
-
Packages that use opennlp.tools.tokenize Package Description opennlp.tools.cmdline.parser opennlp.tools.cmdline.tokenizer opennlp.tools.formats Experimental package related to converting various corpora to OpenNLP Format.opennlp.tools.formats.ad Experimental package related to theArvores Deitadas corpusformat.opennlp.tools.formats.brat Experimental package related to the corpus format used by the "brat rapid annotation tool" (brat).opennlp.tools.formats.conllu Experimental package related to the CoNNL-U format.opennlp.tools.formats.convert opennlp.tools.formats.irishsentencebank Experimental package related to the Irish Sentence Bank format.opennlp.tools.formats.letsmt Experimental package related to theletsmtcorpus format.opennlp.tools.formats.masc Experimental package related to theMASCcorpus format.opennlp.tools.formats.muc Experimental package related to theMUCcorpus format.opennlp.tools.sentdetect Package related to identifying sentence boundaries.opennlp.tools.tokenize Contains classes related to finding token or words in a string.opennlp.tools.tokenize.lang opennlp.tools.tokenize.lang.en opennlp.tools.util.featuregen This package contains classes for generating sequence features. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.parser Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.cmdline.tokenizer Class Description TokenizerEvaluationMonitor A marker interface for evaluatingtokenizers.TokenizerModel TheTokenizerModelis the model used by a learnableTokenizer.TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats Class Description Detokenizer ADetokenizermerges tokens back to their detokenized representation.TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.ad Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.brat Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerModel TheTokenizerModelis the model used by a learnableTokenizer. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.conllu Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.convert Class Description Detokenizer ADetokenizermerges tokens back to their detokenized representation.TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.irishsentencebank Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.letsmt Class Description Detokenizer ADetokenizermerges tokens back to their detokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.masc Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.formats.muc Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens. -
Classes in opennlp.tools.tokenize used by opennlp.tools.sentdetect Class Description Detokenizer ADetokenizermerges tokens back to their detokenized representation. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize Class Description DetokenizationDictionary DetokenizationDictionary.Operation Detokenizer ADetokenizermerges tokens back to their detokenized representation.Detokenizer.DetokenizationOperation This enum contains an operation for every token to merge the tokens together to their detokenized form.SimpleTokenizer A basicTokenizerimplementation which performs tokenization using character classes.TokenContextGenerator Interface for context generators required forTokenizerME.Tokenizer The interface for tokenizers, which segment a string into its tokens.TokenizerEvaluationMonitor A marker interface for evaluatingtokenizers.TokenizerFactory The factory that providesTokenizerdefault implementation and resources.TokenizerModel TheTokenizerModelis the model used by a learnableTokenizer.TokenSample ATokenSampleis text with token spans.WhitespaceTokenizer A basicTokenizerimplementation which performs tokenization using white spaces. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang Class Description TokenContextGenerator Interface for context generators required forTokenizerME. -
Classes in opennlp.tools.tokenize used by opennlp.tools.tokenize.lang.en Class Description TokenSample ATokenSampleis text with token spans. -
Classes in opennlp.tools.tokenize used by opennlp.tools.util.featuregen Class Description Tokenizer The interface for tokenizers, which segment a string into its tokens.