opennlp.tools.tokenize
Class TokenSample
java.lang.Object
opennlp.tools.tokenize.TokenSample
public class TokenSample
- extends java.lang.Object
A TokenSample is text with token spans.
| Methods inherited from class java.lang.Object |
equals, getClass, hashCode, notify, notifyAll, wait, wait, wait |
DEFAULT_SEPARATOR_CHARS
public static final java.lang.String DEFAULT_SEPARATOR_CHARS
- See Also:
- Constant Field Values
TokenSample
public TokenSample(java.lang.String text,
Span[] tokenSpans)
- Initializes the current instance.
- Parameters:
text - the text which contains the tokens.tokenSpans - the spans which mark the begin and end of the tokens.
TokenSample
public TokenSample(Detokenizer detokenizer,
java.lang.String[] tokens)
getText
public java.lang.String getText()
- Retrieves the text.
getTokenSpans
public Span[] getTokenSpans()
- Retrieves the token spans.
toString
public java.lang.String toString()
- Overrides:
toString in class java.lang.Object
parse
public static TokenSample parse(java.lang.String sampleString,
java.lang.String separatorChars)
Copyright © 2011 The Apache Software Foundation. All Rights Reserved.