Uses of Class
com.aliasi.tokenizer.Tokenizer

Packages that use Tokenizer

Package	Description
com.aliasi.test.unit.tokenizer
com.aliasi.tokenizer	Classes for tokenizing character sequences.

Uses of Tokenizer in com.aliasi.test.unit.tokenizer

Subclasses of Tokenizer in com.aliasi.test.unit.tokenizer

Modifier and Type	Class and Description
class	ConstantTokenizer

Methods in com.aliasi.test.unit.tokenizer that return Tokenizer

Modifier and Type	Method and Description
Tokenizer	ConstantTokenizerFactory.tokenizer(char[] cs, int start, int length)
Tokenizer	ConstantTokenizerFactory.transform(String s)

Uses of Tokenizer in com.aliasi.tokenizer

Methods in com.aliasi.tokenizer that return Tokenizer

Modifier and Type	Method and Description
Tokenizer	ModifyTokenTokenizerFactory.modify(Tokenizer tokenizer) Return a modified version of the specified tokenizer that modifies tokens and whitespaces as specified by the corresponding string modifier methods.
protected abstract Tokenizer	ModifiedTokenizerFactory.modify(Tokenizer tokenizer) Return a modified form of the specified tokenizer.
Tokenizer	IndoEuropeanTokenizerFactory.tokenizer(char[] ch, int start, int length) Returns a tokenizer for Indo-European for the specified subsequence of characters.
Tokenizer	RegExTokenizerFactory.tokenizer(char[] cs, int start, int length)
Tokenizer	TokenNGramTokenizerFactory.tokenizer(char[] cs, int start, int len)
Tokenizer	NGramTokenizerFactory.tokenizer(char[] cs, int start, int length) Returns an n-gram tokenizer for the specified characters with the minimum and maximum n-gram lengths as specified in the constructor.
Tokenizer	TokenizerFactory.tokenizer(char[] ch, int start, int length) Returns a tokenizer for the specified subsequence of characters.
Tokenizer	ModifiedTokenizerFactory.tokenizer(char[] cs, int start, int length) Return the tokenizer for the specified character array slice, which is generated by the base tokenizer and modified with the modify method.
Tokenizer	CharacterTokenizerFactory.tokenizer(char[] ch, int start, int length) Returns a character tokenizer for the specified character array slice.

Methods in com.aliasi.tokenizer with parameters of type Tokenizer

Modifier and Type	Method and Description
Tokenizer	ModifyTokenTokenizerFactory.modify(Tokenizer tokenizer) Return a modified version of the specified tokenizer that modifies tokens and whitespaces as specified by the corresponding string modifier methods.
protected abstract Tokenizer	ModifiedTokenizerFactory.modify(Tokenizer tokenizer) Return a modified form of the specified tokenizer.