io.parsingdata.metal.token

Class Token

java.lang.Object

io.parsingdata.metal.token.Token

Direct Known Subclasses:

Cho, Def, Nod, Opt, Pre, Rep, RepN, Seq, Sub, Tie, TokenRef, While

public abstract class Token
extends Object

Base class for all Token implementations.

A Token is the basic building block of a parser. It denotes something that can be parsed, either by reading from the input or composing other tokens.

Specifies the two fields that all tokens share: name (a String) and encoding (an Encoding). A Token is parsed by calling one of the parse methods. Parsing a Token succeeds if the returned ParseResult's succeeded field is true, otherwise parsing has failed.

The name field is used during parsing to construct a scope, by concatenating it to the incoming scope argument as follows:

 scope + SEPARATOR + name
 

The encoding may be null. If it is not, it overrides outer encoding specifications and is passed to nested tokens instead. As such it can itself be overridden by explicit specifications in nested tokens.

Field Summary

Fields

Modifier and Type	Field and Description
Encoding	encoding
String	name
static String	NO_NAME
static String	SEPARATOR

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Token(String name, Encoding encoding)

Method Summary

Modifier and Type	Method and Description
Token	getCanonical(Environment environment)
boolean	isLocal()
protected String	makeNameFragment()
ParseResult	parse(Environment environment, Encoding encoding)
ParseResult	parse(String scope, Environment environment, Encoding encoding)
protected abstract ParseResult	parseImpl(String scope, Environment environment, Encoding encoding)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NO_NAME

public static final String NO_NAME

See Also:

Constant Field Values

SEPARATOR

public static final String SEPARATOR

See Also:

Constant Field Values

name

public final String name

encoding

public final Encoding encoding

Constructor Detail

Token

protected Token(String name,
                Encoding encoding)

Method Detail

parse

public ParseResult parse(String scope,
                         Environment environment,
                         Encoding encoding)
                  throws IOException

Throws:

IOException

parse

public ParseResult parse(Environment environment,
                         Encoding encoding)
                  throws IOException

Throws:

IOException

parseImpl

protected abstract ParseResult parseImpl(String scope,
                                         Environment environment,
                                         Encoding encoding)
                                  throws IOException

Throws:

IOException

isLocal

public boolean isLocal()

getCanonical

public Token getCanonical(Environment environment)

makeNameFragment

protected String makeNameFragment()