Package opennlp.tools.tokenize
Class TokenizerModel
java.lang.Object
opennlp.tools.util.model.BaseModel
opennlp.tools.tokenize.TokenizerModel
- All Implemented Interfaces:
Serializable,ArtifactProvider
The
TokenizerModel is the model used
by a learnable Tokenizer.- See Also:
-
Field Summary
Fields inherited from class opennlp.tools.util.model.BaseModel
TRAINING_CUTOFF_PROPERTY, TRAINING_EVENTHASH_PROPERTY, TRAINING_ITERATIONS_PROPERTY -
Constructor Summary
ConstructorsConstructorDescriptionTokenizerModel(File modelFile) Initializes aTokenizerModelinstance via a validFile.Initializes aTokenizerModelinstance via a validInputStream.TokenizerModel(URL modelURL) Initializes aTokenizerModelinstance via a validURL.TokenizerModel(Path modelPath) Initializes aTokenizerModelinstance via a validPath.TokenizerModel(MaxentModel tokenizerModel, Map<String, String> manifestInfoEntries, TokenizerFactory tokenizerFactory) Initializes aTokenizerModelinstance via aMaxentModeland related resources. -
Method Summary
Methods inherited from class opennlp.tools.util.model.BaseModel
getArtifact, getLanguage, getManifestProperty, getVersion, isLoadedFromSerialized, serialize, serialize, serialize
-
Constructor Details
-
TokenizerModel
public TokenizerModel(MaxentModel tokenizerModel, Map<String, String> manifestInfoEntries, TokenizerFactory tokenizerFactory) Initializes aTokenizerModelinstance via aMaxentModeland related resources.- Parameters:
tokenizerModel- Themodelto be used.manifestInfoEntries- Additional information kept in the manifest.tokenizerFactory- TheTokenizerFactoryto be used internally.
-
TokenizerModel
Initializes aTokenizerModelinstance via a validInputStream.- Parameters:
in- TheInputStreamused for loading the model.- Throws:
IOException- Thrown if IO errors occurred during initialization.
-
TokenizerModel
Initializes aTokenizerModelinstance via a validFile.- Parameters:
modelFile- TheFileused for loading the model.- Throws:
IOException- Thrown if IO errors occurred during initialization.
-
TokenizerModel
Initializes aTokenizerModelinstance via a validPath.- Parameters:
modelPath- ThePathused for loading the model.- Throws:
IOException- Thrown if IO errors occurred during initialization.
-
TokenizerModel
Initializes aTokenizerModelinstance via a validURL.- Parameters:
modelURL- TheURLused for loading the model.- Throws:
IOException- Thrown if IO errors occurred during initialization.
-
-
Method Details
-
getFactory
- Returns:
- Retrieves the active
TokenizerFactory.
-
getMaxentModel
- Returns:
- Retrieves the model as
MaxentModelinstance.
-
getAbbreviations
- Returns:
- Retrieves the active abbreviation
Dictionary.
-
useAlphaNumericOptimization
public boolean useAlphaNumericOptimization()- Returns:
trueif alphanumeric optimization is active,falseotherwise.
-
hashCode
public int hashCode() -
equals
-