Uses of Class
net.sf.okapi.steps.tokenization.common.Lexem

Packages that use Lexem

Package	Description
net.sf.okapi.steps.tokenization.common
net.sf.okapi.steps.tokenization.engine

Uses of Lexem in net.sf.okapi.steps.tokenization.common

Methods in net.sf.okapi.steps.tokenization.common that return Lexem

Modifier and Type	Method and Description
Lexem	Token.getLexem()
protected abstract Lexem	AbstractLexer.lexer_next()
Lexem	AbstractLexer.next()
Lexem	ILexer.next() Gets the next lexem available.

Constructors in net.sf.okapi.steps.tokenization.common with parameters of type Lexem

Constructor and Description
Token(int tokenId, Lexem lexem, int score)

Uses of Lexem in net.sf.okapi.steps.tokenization.engine

Methods in net.sf.okapi.steps.tokenization.engine that return Lexem

Modifier and Type	Method and Description
protected Lexem	Sorter.lexer_next()
protected Lexem	TokenScanner.lexer_next()
protected Lexem	TextScanner.lexer_next()
Lexem	Retainer.lexer_next()
protected Lexem	GarbageCollector.lexer_next()
Lexem	Remover.lexer_next()
protected Lexem	LookUpLexer.lexer_next()
protected Lexem	TokenMatcher.lexer_next()
protected Lexem	JavaCcLexer.lexer_next()
Lexem	RbbiLexer.lexer_next()
protected Lexem	Reconciler.lexer_next()