io.parsingdata.metal.token

Class Token

java.lang.Object

io.parsingdata.metal.token.Token

Direct Known Subclasses:

Cho, Def, Post, Pre, Rep, RepN, Seq, Sub, Tie, TokenRef, Until, While

public abstract class Token
extends Object

Base class for all Token implementations.

A Token is the basic building block of a parser. It denotes something that can be parsed, either by reading from the input or composing other tokens.

Specifies the two fields that all tokens share: name (a String) and encoding (an Encoding). A Token is parsed by calling one of the parse methods. Parsing a Token succeeds if the returned Optional contains an ParseState, otherwise parsing has failed.

The name field is used during parsing to construct a scope, by concatenating it to the incoming scope argument as follows:

 scope + SEPARATOR + name
 

The encoding may be null. If it is not, it overrides outer encoding specifications and is passed to nested tokens instead. As such it can itself be overridden by explicit specifications in nested tokens.

Field Summary

Fields

Modifier and Type	Field and Description
static String	EMPTY_NAME
Encoding	encoding
String	name
static String	NO_NAME
static String	SEPARATOR

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Token(String name, Encoding encoding)

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object obj)
Token	getCanonical(ParseState parseState)
int	hashCode()
boolean	isLocal()
protected String	makeNameFragment()
Optional<ParseState>	parse(Environment environment)
protected abstract Optional<ParseState>	parseImpl(Environment environment)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

NO_NAME

public static final String NO_NAME

See Also:

Constant Field Values

SEPARATOR

public static final String SEPARATOR

See Also:

Constant Field Values

EMPTY_NAME

public static final String EMPTY_NAME

See Also:

Constant Field Values

name

public final String name

encoding

public final Encoding encoding

Constructor Detail

Token

protected Token(String name,
                Encoding encoding)

Method Detail

parse

public Optional<ParseState> parse(Environment environment)

parseImpl

protected abstract Optional<ParseState> parseImpl(Environment environment)

isLocal

public boolean isLocal()

getCanonical

public Token getCanonical(ParseState parseState)

makeNameFragment

protected String makeNameFragment()

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object