public class WhitespaceNormTokenizerFactory extends ModifyTokenTokenizerFactory implements Serializable
WhitespaceNormTokenizerFactory filters the tokenizers produced
by a base tokenizer factory to convert non-empty whitespaces to a single
space and leave empty (zero-length) whitespaces alone.
| Constructor and Description |
|---|
WhitespaceNormTokenizerFactory(TokenizerFactory factory)
Construct a whitespace normalizing tokenizer factory from the
specified base factory.
|
| Modifier and Type | Method and Description |
|---|---|
String |
modifyWhitespace(String whitespace)
Return the normalized form of the specified whitespace.
|
String |
toString() |
modify, modifyTokenbaseTokenizerFactory, tokenizerpublic WhitespaceNormTokenizerFactory(TokenizerFactory factory)
factory - Base tokenizer factory.public String modifyWhitespace(String whitespace)
modifyWhitespace in class ModifyTokenTokenizerFactorywhitespace - Input whitespace.public String toString()
toString in class ModifyTokenTokenizerFactoryCopyright © 2019 Alias-i, Inc.. All rights reserved.