ru.histone.v2.parser.tokenizer

Class Tokenizer

java.lang.Object

ru.histone.v2.parser.tokenizer.Tokenizer

public class Tokenizer
extends Object

Histone template tokenizer
Tokenizer is based on regular expressions and returns tokens one by one. All tokens are configured in Tokenizer

Author:

Alexey Nevinsky

Constructor Summary

Constructors

Constructor and Description
Tokenizer(String input, String baseURI, List<Expression> expressions)

Method Summary

Modifier and Type	Method and Description
String	getBaseURI()
int	getLineNumber(long index)
Token	getTokenFromBuffer(int offset, List<Integer> ignored)
TokenizerResult	next(List<Integer> ignored, Integer... selector)
void	readTokenToBuffer()
Tokenizer	setBaseURI(String baseURI)
TokenizerResult	test(List<Integer> ignored, Integer... selector)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Tokenizer

public Tokenizer(String input,
                 String baseURI,
                 List<Expression> expressions)

Method Detail

getBaseURI

public String getBaseURI()

setBaseURI

public Tokenizer setBaseURI(String baseURI)

readTokenToBuffer

public void readTokenToBuffer()

getTokenFromBuffer

public Token getTokenFromBuffer(int offset,
                                List<Integer> ignored)

next

public TokenizerResult next(List<Integer> ignored,
                            Integer... selector)

test

public TokenizerResult test(List<Integer> ignored,
                            Integer... selector)

getLineNumber

public int getLineNumber(long index)