Package banner.tokenization

Interface Summary

Interface	Description
Tokenizer	Tokenizers take the text of a sentence and turn it into a Sentence object.

Class Summary

Class	Description
SimpleTokenizer	Tokens ouput by this tokenizer consist of a contiguous block of alphanumeric characters or a single punctuation mark.