regexodus

Class RETokenizer

java.lang.Object

regexodus.RETokenizer

All Implemented Interfaces:

Iterator<String>

public class RETokenizer
extends Object
implements Iterator<String>

The Tokenizer class suggests a methods to break a text into tokens using occurrences of a pattern as delimiters. There are two ways to obtain a text tokenizer for some pattern:

 Pattern p=new Pattern("\\s+"); //any number of space characters
 String text="blah blah blah";
 //by factory method
 RETokenizer tok1=p.tokenizer(text);
 //or by constructor
 RETokenizer tok2=new RETokenizer(p,text);
 

Now the one way is to use the tokenizer as a token enumeration/iterator:

 while(tok1.hasMore()) System.out.println(tok1.nextToken());
 

and another way is to split it into a String array: String[] arr=tok2.split(); for(int i=0;i<tok2.length;i++) System.out.println(arr[i]);

See Also:

Pattern.tokenizer(java.lang.String)

Constructor Summary

Constructors

Constructor and Description
RETokenizer(Pattern pattern, char[] chars, int off, int len)
RETokenizer(Pattern pattern, Reader r, int len)
RETokenizer(Pattern pattern, String text)

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object o)
int	hashCode()
boolean	hasNext()
boolean	isEmptyEnabled()
String	next()
void	remove() Removes from the underlying collection the last element returned by this iterator (optional operation).
void	reset()
void	setEmptyEnabled(boolean b)
String[]	split()
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Constructor Detail

RETokenizer

public RETokenizer(Pattern pattern,
                   String text)

RETokenizer

public RETokenizer(Pattern pattern,
                   char[] chars,
                   int off,
                   int len)

RETokenizer

public RETokenizer(Pattern pattern,
                   Reader r,
                   int len)
            throws IOException

Throws:

IOException

Method Detail

setEmptyEnabled

public void setEmptyEnabled(boolean b)

isEmptyEnabled

public boolean isEmptyEnabled()

split

public String[] split()

reset

public void reset()

remove

public void remove()

Removes from the underlying collection the last element returned by this iterator (optional operation). This method can be called only once per call to next(). The behavior of an iterator is unspecified if the underlying collection is modified while the iteration is in progress in any way other than by calling this method.

Specified by:

remove in interface Iterator<String>

Throws:

UnsupportedOperationException - if the remove operation is not supported by this iterator

IllegalStateException - if the next method has not yet been called, or the remove method has already been called after the last call to the next method

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<String>

next

public String next()

Specified by:

next in interface Iterator<String>

Returns:

a next token as a String

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object