TokenType Enum
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Specifies the different types of tokens that can be identified and returned from a language service scanner.
public enum class TokenTypepublic enum class TokenTypeenum TokenTypepublic enum TokenTypetype TokenType = Public Enum TokenType- Inheritance
-
TokenType
Fields
| Comment | 10 | A block comment. For example, in C# or C++, a comment is bounded by /* and */. In XML, the comment is bounded by <!-- and -->. |
| Delimiter | 7 | A token that operates as a separator between two language elements. For example, in C#, the period "." between class name and member name. In XML, the angle brackets surrounding a tag, < and >. |
| Identifier | 3 | An identifier or name. For example, the name of a variable, method, or class. In XML, this could be the name of a tag or attribute. |
| Keyword | 2 | A language keyword, an identifier that is reserved by the language. For example, in C#, |
| LineComment | 9 | A line comment (comment is terminated at the end of the line). For example, in C# or C++, a comment is preceded by a //. In Visual Basic, this is a single tick '. |
| Literal | 5 | A literal value (a character or number). For example, in C# or C++, this is a character bounded by single quotes, or a decimal or hexadecimal number. |
| Operator | 6 | A punctuation character that has a specific meaning in a language. For example, in C#, arithmetic operators +, -, *, and /. In C++, pointer dereference operator ->, insertion operator >>, and extraction operation <<. In XML, assignment operator =. |
| String | 4 | A string. Typically defined as zero or more characters bounded by double quotes. |
| Text | 1 | General text; any text not identified as a specified token type. |
| Unknown | 0 | The token is an unknown type. This is typically used for any token not recognized by the parser and should be considered an error in the code being parsed. |
| WhiteSpace | 8 | A space, tab, or newline. Typically, a contiguous run of any whitespace is considered a single whitespace token. For example, the three spaces in "name this" would be treated as one whitespace token. |
Remarks
This enumeration is used in the TokenType structure to identify the type of token parsed. The TokenType structure is used in the IScanner scanner as implemented in a language service.
The types specified in this enumeration cover tokens that can appear in all common computer languages. More importantly, these are the token types that the default managed package framework language service classes understand. You can add additional types that your scanner support but you should first adhere to the meanings of the types described here.