Uses of Class
opennlp.tools.tokenize.TokenizerFactory
Packages that use TokenizerFactory
Package
Description
Contains classes related to finding token or words in a string.
-
Uses of TokenizerFactory in opennlp.tools.tokenize
Methods in opennlp.tools.tokenize that return TokenizerFactoryModifier and TypeMethodDescriptionstatic TokenizerFactoryTokenizerFactory.create(String subclassName, String languageCode, Dictionary abbreviationDictionary, boolean useAlphaNumericOptimization, Pattern alphaNumericPattern) Factory method the framework uses instantiate a newTokenizerFactory.TokenizerModel.getFactory()Methods in opennlp.tools.tokenize with parameters of type TokenizerFactoryModifier and TypeMethodDescriptionstatic TokenizerModelTokenizerME.train(ObjectStream<TokenSample> samples, TokenizerFactory factory, TrainingParameters mlParams) Trains a model for theTokenizerME.Constructors in opennlp.tools.tokenize with parameters of type TokenizerFactoryModifierConstructorDescriptionTokenizerCrossValidator(TrainingParameters params, TokenizerFactory factory, TokenizerEvaluationMonitor... listeners) Creates aTokenizerCrossValidatorusing the givenTokenizerFactory.TokenizerModel(MaxentModel tokenizerModel, Map<String, String> manifestInfoEntries, TokenizerFactory tokenizerFactory) Initializes aTokenizerModelinstance via aMaxentModeland related resources.