Package opennlp.tools.util.featuregen
Class TokenPatternFeatureGenerator
- java.lang.Object
-
- opennlp.tools.util.featuregen.TokenPatternFeatureGenerator
-
- All Implemented Interfaces:
AdaptiveFeatureGenerator
public class TokenPatternFeatureGenerator extends Object implements AdaptiveFeatureGenerator
Partitions tokens into sub-tokens based on character classes and generates class features for each of the sub-tokens and combinations of those sub-tokens.
-
-
Constructor Summary
Constructors Constructor Description TokenPatternFeatureGenerator()Initializes aTokenPatternFeatureGenerator.TokenPatternFeatureGenerator(Tokenizer supportTokenizer)Initializes aTokenPatternFeatureGeneratorinstance.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidcreateFeatures(List<String> feats, String[] toks, int index, String[] preds)Adds the appropriate features for the token at the specifiedindexwith the specified array ofpreviousOutcomesto the specified list of features.-
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface opennlp.tools.util.featuregen.AdaptiveFeatureGenerator
clearAdaptiveData, updateAdaptiveData
-
-
-
-
Constructor Detail
-
TokenPatternFeatureGenerator
public TokenPatternFeatureGenerator()
Initializes aTokenPatternFeatureGenerator. For tokenization theSimpleTokenizeris used.
-
TokenPatternFeatureGenerator
public TokenPatternFeatureGenerator(Tokenizer supportTokenizer)
Initializes aTokenPatternFeatureGeneratorinstance.- Parameters:
supportTokenizer- TheTokenizerto be used.
-
-
Method Detail
-
createFeatures
public void createFeatures(List<String> feats, String[] toks, int index, String[] preds)
Description copied from interface:AdaptiveFeatureGeneratorAdds the appropriate features for the token at the specifiedindexwith the specified array ofpreviousOutcomesto the specified list of features.- Specified by:
createFeaturesin interfaceAdaptiveFeatureGenerator- Parameters:
feats- The list of features to be added to.toks- The tokens of the sentence or other text unit being processed.index- The index of the token which is currently being processed.preds- The outcomes for the tokens prior to the specified index.
-
-