com.aliasi.tokenizer

Interface TokenCategorizer

All Known Implementing Classes:

CharacterTokenCategorizer, IndoEuropeanTokenCategorizer

public interface TokenCategorizer

A TokenCategorizer supplies a string-based category for string-based tokens.

Since:

LingPipe1.0

Version:

1.0

Author:

Bob Carpenter

Method Summary

Methods

Modifier and Type	Method and Description
String[]	categories() Returns an array of all available categories produced by this categorizer.
String	categorize(String token) Returns the category string for the specified token.

Method Detail

categorize

String categorize(String token)

Returns the category string for the specified token.

Parameters:

token - Token to categorize.

Returns:

Category of token.

categories

String[] categories()

Returns an array of all available categories produced by this categorizer.

Returns:

Array of categories produced by this categorizer.