Okapi Step for Tokenization 0.35 API

Packages

Package	Description
net.sf.okapi.steps.tokenization
net.sf.okapi.steps.tokenization.common
net.sf.okapi.steps.tokenization.engine
net.sf.okapi.steps.tokenization.engine.javacc
net.sf.okapi.steps.tokenization.locale
net.sf.okapi.steps.tokenization.tokens