Uses of Interface
banner.tokenization.Tokenizer

Packages that use Tokenizer

Package	Description
banner.eval
banner.eval.dataset
banner.normalization
banner.postprocessing
banner.tagging.dictionary
banner.tokenization
de.julielab.jcore.banner.dataset

Uses of Tokenizer in banner.eval

Methods in banner.eval that return Tokenizer

Modifier and Type	Method	Description
static Tokenizer	BANNER.getTokenizer(org.apache.commons.configuration.HierarchicalConfiguration config)

Methods in banner.eval with parameters of type Tokenizer

Modifier and Type	Method	Description
static Sentence	BANNER.process(Tagger tagger, Tokenizer tokenizer, PostProcessor postProcessor, Sentence sentence)

Uses of Tokenizer in banner.eval.dataset

Fields in banner.eval.dataset declared as Tokenizer

Modifier and Type	Field	Description
protected Tokenizer	Dataset.tokenizer

Methods in banner.eval.dataset with parameters of type Tokenizer

Modifier and Type	Method	Description
protected Sentence	AZDCDataset.getSentence(String sentenceId, String documentId, String sentenceText, Tokenizer tokenizer, HashMap<String,LinkedList<Dataset.Tag>> tags, Map<String,Set<String>> nameToId, Map<String,Set<String>> idToName)
protected Sentence	BC2GMDataset.getSentence(String id, String sentenceText, Tokenizer tokenizer, HashMap<String,LinkedList<Dataset.Tag>> tags)
void	Dataset.setTokenizer(Tokenizer tokenizer)

Constructors in banner.eval.dataset with parameters of type Tokenizer

Constructor	Description
AZDCDataset(Tokenizer tokenizer)
BC2GMDataset(Tokenizer tokenizer)

Uses of Tokenizer in banner.normalization

Methods in banner.normalization with parameters of type Tokenizer

Modifier and Type	Method	Description
void	Gazetteer.load(Tokenizer tokenizer, EntityType type, String filename)
void	Gazetteer.load(Tokenizer tokenizer, String namesFilename)
void	Gazetteer.load(Tokenizer tokenizer, String namesFilename, Gazetteer.EntityFilter entityFilter)

Uses of Tokenizer in banner.postprocessing

Constructors in banner.postprocessing with parameters of type Tokenizer

Constructor	Description
OneSensePerDocument(Tokenizer tokenizer, boolean createNew)

Uses of Tokenizer in banner.tagging.dictionary

Methods in banner.tagging.dictionary that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	DictionaryTagger.getTokenizer()

Methods in banner.tagging.dictionary with parameters of type Tokenizer

Modifier and Type	Method	Description
void	DictionaryTagger.configure(org.apache.commons.configuration.HierarchicalConfiguration config, Tokenizer tokenizer)
void	DictionaryTagger.setTokenizer(Tokenizer tokenizer)

Uses of Tokenizer in banner.tokenization

Classes in banner.tokenization that implement Tokenizer

Modifier and Type	Class	Description
class	SimpleTokenizer	Tokens ouput by this tokenizer consist of a contiguous block of alphanumeric characters or a single punctuation mark.

Uses of Tokenizer in de.julielab.jcore.banner.dataset

Constructors in de.julielab.jcore.banner.dataset with parameters of type Tokenizer

Constructor	Description
JCoReEntityDataset(Tokenizer tokenizer)