|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||
java.lang.Objectopennlp.tools.tokenize.WhitespaceTokenizer
public class WhitespaceTokenizer
This tokenizer uses white spaces to tokenize the input text.
To obtain an instance of this tokenizer use the static final
INSTANCE field.
| Field Summary | |
|---|---|
static WhitespaceTokenizer |
INSTANCE
Use this static reference to retrieve an instance of the WhitespaceTokenizer. |
| Method Summary | |
|---|---|
String[] |
tokenize(String s)
Splits a string into its atomic parts |
Span[] |
tokenizePos(String d)
Finds the boundaries of atomic parts in a string. |
| Methods inherited from class java.lang.Object |
|---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Field Detail |
|---|
public static final WhitespaceTokenizer INSTANCE
WhitespaceTokenizer.
| Method Detail |
|---|
public Span[] tokenizePos(String d)
Tokenizer
d - The string to be tokenized.
public String[] tokenize(String s)
Tokenizer
tokenize in interface Tokenizers - The string to be tokenized.
|
|||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | ||||||||