Uses of Interface
de.jungblut.nlp.Tokenizer

Packages that use Tokenizer

Package	Description
de.jungblut.nlp

Uses of Tokenizer in de.jungblut.nlp

Classes in de.jungblut.nlp that implement Tokenizer

Modifier and Type	Class	Description
class	BigramTokenizer	Advanced tokenizer that lowercases, adds start and end tags, deduplicates tokens and builds bigrams.
class	StandardTokenizer	Just a basic tokenizer by certain attributes with normalization.