net.sf.okapi.steps.tokenization

Class RbbiTokenizer

java.lang.Object

net.sf.okapi.steps.tokenization.RbbiTokenizer

All Implemented Interfaces:

ITokenizer

public class RbbiTokenizer
extends Object
implements ITokenizer

Nested Class Summary

Nested classes/interfaces inherited from interface net.sf.okapi.steps.tokenization.ITokenizer

ITokenizer.TokenType

Constructor Summary

Constructors

Constructor and Description
RbbiTokenizer()

Method Summary

Modifier and Type	Method and Description
boolean	hasNext()
void	init(String text, LocaleId language)
Token	next()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RbbiTokenizer

public RbbiTokenizer()

Method Detail

hasNext

public boolean hasNext()

Specified by:

hasNext in interface ITokenizer

next

public Token next()

Specified by:

next in interface ITokenizer

init

public void init(String text,
                 LocaleId language)

Specified by:

init in interface ITokenizer