public class TokenSample extends Object
TokenSample is text with token spans.| Modifier and Type | Field and Description |
|---|---|
static String |
DEFAULT_SEPARATOR_CHARS |
| Constructor and Description |
|---|
TokenSample(Detokenizer detokenizer,
String[] tokens) |
TokenSample(String text,
Span[] tokenSpans)
Initializes the current instance.
|
| Modifier and Type | Method and Description |
|---|---|
boolean |
equals(Object obj) |
String |
getText()
Retrieves the text.
|
Span[] |
getTokenSpans()
Retrieves the token spans.
|
int |
hashCode() |
static TokenSample |
parse(String sampleString,
String separatorChars) |
String |
toString() |
public static final String DEFAULT_SEPARATOR_CHARS
public TokenSample(String text, Span[] tokenSpans)
text - the text which contains the tokens.tokenSpans - the spans which mark the begin and end of the tokens.public TokenSample(Detokenizer detokenizer, String[] tokens)
public String getText()
public Span[] getTokenSpans()
public static TokenSample parse(String sampleString, String separatorChars)
Copyright © 2017 The Apache Software Foundation. All rights reserved.