net.sf.okapi.steps.tokenization.engine.javacc

Interface WordTokenizerConstants

All Known Implementing Classes:

WordTokenizer, WordTokenizerTokenManager

public interface WordTokenizerConstants

Token literal values and constants. Generated by org.javacc.parser.OtherFilesGen#start()

Field Summary

Fields

Modifier and Type	Field and Description
static int	ACRONYM RegularExpression Id.
static int	ALPHA RegularExpression Id.
static int	ALPHANUM RegularExpression Id.
static int	APOSTROPHE RegularExpression Id.
static int	COMPANY RegularExpression Id.
static int	DEFAULT Lexical state.
static int	DIGIT RegularExpression Id.
static int	EMAIL RegularExpression Id.
static int	EOF End of File.
static int	HAS_DIGIT RegularExpression Id.
static int	HOST RegularExpression Id.
static int	HYPHENATED RegularExpression Id.
static int	LETTER RegularExpression Id.
static int	NOISE RegularExpression Id.
static int	NUM RegularExpression Id.
static int	P RegularExpression Id.
static String[]	tokenImage Literal token values.

Field Detail

EOF

static final int EOF

End of File.

See Also:

Constant Field Values

HYPHENATED

static final int HYPHENATED

RegularExpression Id.

See Also:

Constant Field Values

ALPHANUM

static final int ALPHANUM

RegularExpression Id.

See Also:

Constant Field Values

APOSTROPHE

static final int APOSTROPHE

RegularExpression Id.

See Also:

Constant Field Values

ACRONYM

static final int ACRONYM

RegularExpression Id.

See Also:

Constant Field Values

COMPANY

static final int COMPANY

RegularExpression Id.

See Also:

Constant Field Values

EMAIL

static final int EMAIL

RegularExpression Id.

See Also:

Constant Field Values

HOST

static final int HOST

RegularExpression Id.

See Also:

Constant Field Values

NUM

static final int NUM

RegularExpression Id.

See Also:

Constant Field Values

P

static final int P

RegularExpression Id.

See Also:

Constant Field Values

HAS_DIGIT

static final int HAS_DIGIT

RegularExpression Id.

See Also:

Constant Field Values

ALPHA

static final int ALPHA

RegularExpression Id.

See Also:

Constant Field Values

LETTER

static final int LETTER

RegularExpression Id.

See Also:

Constant Field Values

DIGIT

static final int DIGIT

RegularExpression Id.

See Also:

Constant Field Values

NOISE

static final int NOISE

RegularExpression Id.

See Also:

Constant Field Values

DEFAULT

static final int DEFAULT

Lexical state.

See Also:

Constant Field Values

tokenImage

static final String[] tokenImage

Literal token values.