Token (HyperGraphDB Prolog 1.3-SNAPSHOT API)

java.lang.Object
- alice.util.jedit.Token

```
public class Token
extends Object
```
A linked list of tokens. Each token has three fields - a token identifier, which is a byte value that can be looked up in the array returned by SyntaxDocument.getColors() to get a color value, a length value which is the length of the token in the text, and a pointer to the next token in the list.

Author:

Slava Pestov

Field Summary

Fields
Modifier and Type	Field and Description
`static byte`	`COMMENT1` Comment 1 token id.
`static byte`	`COMMENT2` Comment 2 token id.
`static byte`	`END` The token type, that along with a length of 0 marks the end of the token list.
`byte`	`id` The id of this token.
`static byte`	`ID_COUNT` The total number of defined token ids.
`static byte`	`INTERNAL_FIRST` The first id that can be used for internal state in a token marker.
`static byte`	`INTERNAL_LAST` The last id that can be used for internal state in a token marker.
`static byte`	`INVALID` Invalid token id.
`static byte`	`KEYWORD1` Keyword 1 token id.
`static byte`	`KEYWORD2` Keyword 2 token id.
`static byte`	`KEYWORD3` Keyword 3 token id.
`static byte`	`LABEL` Label token id.
`int`	`length` The length of this token.
`static byte`	`LITERAL1` Literal 1 token id.
`static byte`	`LITERAL2` Literal 2 token id.
`Token`	`next` The next token in the linked list.
`static byte`	`NULL` Normal text token id.
`static byte`	`OPERATOR` Operator token id.

Constructor Summary

Constructors
Constructor and Description

Token(int length, byte id)
Creates a new token.

Constructors
Constructor and Description
`Token(int length, byte id)` Creates a new token.

Method Summary

Methods
Modifier and Type Method and Description

String toString()
Returns a string representation of this token.
- Methods inherited from class java.lang.Object
  clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods
Modifier and Type	Method and Description
`String`	`toString()` Returns a string representation of this token.

- Field Detail
  - NULL
```
public static final byte NULL
```
    Normal text token id. This should be used to mark normal text.
    
    See Also:
    Constant Field Values
  - COMMENT1
```
public static final byte COMMENT1
```
    Comment 1 token id. This can be used to mark a comment.
    
    See Also:
    Constant Field Values
  - COMMENT2
```
public static final byte COMMENT2
```
    Comment 2 token id. This can be used to mark a comment.
    
    See Also:
    Constant Field Values
  - LITERAL1
```
public static final byte LITERAL1
```
    Literal 1 token id. This can be used to mark a string literal (eg, C mode uses this to mark "..." literals)
    
    See Also:
    Constant Field Values
  - LITERAL2
```
public static final byte LITERAL2
```
    Literal 2 token id. This can be used to mark an object literal (eg, Java mode uses this to mark true, false, etc)
    
    See Also:
    Constant Field Values
  - LABEL
```
public static final byte LABEL
```
    Label token id. This can be used to mark labels (eg, C mode uses this to mark ...: sequences)
    
    See Also:
    Constant Field Values
  - KEYWORD1
```
public static final byte KEYWORD1
```
    Keyword 1 token id. This can be used to mark a keyword. This should be used for general language constructs.
    
    See Also:
    Constant Field Values
  - KEYWORD2
```
public static final byte KEYWORD2
```
    Keyword 2 token id. This can be used to mark a keyword. This should be used for preprocessor commands, or variables.
    
    See Also:
    Constant Field Values
  - KEYWORD3
```
public static final byte KEYWORD3
```
    Keyword 3 token id. This can be used to mark a keyword. This should be used for data types.
    
    See Also:
    Constant Field Values
  - OPERATOR
```
public static final byte OPERATOR
```
    Operator token id. This can be used to mark an operator. (eg, SQL mode marks +, -, etc with this token type)
    
    See Also:
    Constant Field Values
  - INVALID
```
public static final byte INVALID
```
    Invalid token id. This can be used to mark invalid or incomplete tokens, so the user can easily spot syntax errors.
    
    See Also:
    Constant Field Values
  - ID_COUNT
```
public static final byte ID_COUNT
```
    The total number of defined token ids.
    
    See Also:
    Constant Field Values
  - INTERNAL_FIRST
```
public static final byte INTERNAL_FIRST
```
    The first id that can be used for internal state in a token marker.
    
    See Also:
    Constant Field Values
  - INTERNAL_LAST
```
public static final byte INTERNAL_LAST
```
    The last id that can be used for internal state in a token marker.
    
    See Also:
    Constant Field Values
  - END
```
public static final byte END
```
    The token type, that along with a length of 0 marks the end of the token list.
    
    See Also:
    Constant Field Values
  - length
```
public int length
```
    The length of this token.
  - id
```
public byte id
```
    The id of this token.
  - next
```
public Token next
```
    The next token in the linked list.
- Constructor Detail
  - Token
```
public Token(int length,
     byte id)
```
    Creates a new token.
    
    Parameters:
    length - The length of the token
    id - The id of the token
- Method Detail
  - toString
```
public String toString()
```
    Returns a string representation of this token.
    
    Overrides:
    
    toString in class Object

Class Token

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

NULL

COMMENT1

COMMENT2

LITERAL1

LITERAL2

LABEL

KEYWORD1

KEYWORD2

KEYWORD3

OPERATOR

INVALID

ID_COUNT

INTERNAL_FIRST

INTERNAL_LAST

END

length

id

next

Constructor Detail

Token

Method Detail

toString