opennlp.tools.tokenize
Class TokenSample
java.lang.Object
opennlp.tools.tokenize.TokenSample
public class TokenSample
- extends Object
A TokenSample is text with token spans.
DEFAULT_SEPARATOR_CHARS
public static final String DEFAULT_SEPARATOR_CHARS
- See Also:
- Constant Field Values
TokenSample
public TokenSample(String text,
Span[] tokenSpans)
- Initializes the current instance.
- Parameters:
text - the text which contains the tokens.tokenSpans - the spans which mark the begin and end of the tokens.
TokenSample
public TokenSample(Detokenizer detokenizer,
String[] tokens)
getText
public String getText()
- Retrieves the text.
getTokenSpans
public Span[] getTokenSpans()
- Retrieves the token spans.
toString
public String toString()
- Overrides:
toString in class Object
parse
public static TokenSample parse(String sampleString,
String separatorChars)
equals
public boolean equals(Object obj)
- Overrides:
equals in class Object
Copyright © 2013 The Apache Software Foundation. All Rights Reserved.