Package de.jplag

Class Token

java.lang.Object
de.jplag.Token
public class Token extends Object
This class represents a token in a source code. It can represent keywords, identifiers, syntactical structures etc. What types of tokens there are depends on the specific language, meaning JPlag does not enforce a specific token set. The language parsers decide what is a token and what is not.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    NO_VALUE
    Indicates that the requested field has no value.

  • Constructor Summary

    Constructors
    Constructor
    Description
    Token(TokenType type, File file, int line, int column, int length)
    Creates a token with column and length information.
    Token(TokenType type, File file, int line, int column, int length, CodeSemantics semantics)
    Creates a token with column, length and semantic information.
    Token(TokenType type, File file, TokenTrace trace)
    Creates a token with column and length information.

  • Method Summary

    Modifier and Type
    Method
    Description
    static Token
    fileEnd(File file)
    Creates a token of type FILE_END without information about line, column, and length.
    int
    getColumn()
    Returns the character index which denotes where the code sections represented by this token starts in the line.
    File
    getFile()
     
    int
    getLength()
    Gives the length if the code sections represented by this token.
    int
    getLine()
    Gives the line index denoting in which line the code sections represented by this token starts.
    CodeSemantics
    getSemantics()
     
    TokenType
    getType()
     
    static Token
    semanticFileEnd(File file)
    Creates a token of type FILE_END without information about line, column, and length, but with semantic information.
    String
    toString()
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

    • NO_VALUE

      public static final int NO_VALUE
      Indicates that the requested field has no value.
      See Also:

  • Constructor Details

    • Token

      public Token(TokenType type, File file, int line, int column, int length)
      Creates a token with column and length information.
      Parameters:
      type - is the token type.
      file - is the name of the source code file.
      line - is the line index in the source code where the token resides. Index is 1-based.
      column - is the column index, meaning where the token starts in the line. Index is 1-based.
      length - is the length of the token in the source code.

    • Token

      public Token(TokenType type, File file, TokenTrace trace)
      Creates a token with column and length information.
      Parameters:
      type - is the token type.
      file - is the name of the source code file.
      trace - is the tracing information of the token, meaning line, column, and length.

    • Token

      public Token(TokenType type, File file, int line, int column, int length, CodeSemantics semantics)
      Creates a token with column, length and semantic information.
      Parameters:
      type - is the token type.
      file - is the name of the source code file.
      line - is the line index in the source code where the token resides. Index is 1-based.
      column - is the column index, meaning where the token starts in the line. Index is 1-based.
      length - is the length of the token in the source code.
      semantics - is a record containing semantic information about the token.

  • Method Details

    • fileEnd

      public static Token fileEnd(File file)
      Creates a token of type FILE_END without information about line, column, and length.
      Parameters:
      file - is the name of the source code file.

    • semanticFileEnd

      public static Token semanticFileEnd(File file)
      Creates a token of type FILE_END without information about line, column, and length, but with semantic information.
      Parameters:
      file - is the name of the source code file.

    • getColumn

      public int getColumn()
      Returns the character index which denotes where the code sections represented by this token starts in the line.
      Returns:
      the character index in the line.

    • getFile

      public File getFile()
      Returns:
      the name of the file where the source code that the token represents is located in.

    • getLength

      public int getLength()
      Gives the length if the code sections represented by this token.
      Returns:
      the length in characters.

    • getLine

      public int getLine()
      Gives the line index denoting in which line the code sections represented by this token starts.
      Returns:
      the line index.

    • getType

      public TokenType getType()
      Returns:
      the type of the token.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getSemantics

      public CodeSemantics getSemantics()
      Returns:
      the semantics of the token.