Metadata Tokens Overview
Metadata refers to declarative information about abstractions, including runtime types (classes, value types, and interfaces), global functions, and global variables. The metadata is stored in tables — one table for each category of abstraction and one row of the table for each declaration of an abstraction. A token (an object of type mdToken) is used to locate the record that contains the metadata for an abstraction. The metadata engine uses the token to index into a specific metadata table in a given metadata scope.
Metadata Token Structure
A metadata token is a 4-byte value. The most significant byte (MSB) specifies the token type and consequently identifies the abstraction and its associated metadata table. For example, a value of 1 in the MSB means that the token is an mdTypeRef token, which represents a type reference, and that its metadata is stored in the TypeRef metadata table; a value of 4 in the MSB corresponds to an mdFieldDef token. The CorTokenType enumeration is used to specify the token types.
The lower three bytes, referred to as the record identifier (RID), contain the index of the row within the metadata table to which the token's MSB refers. For example, the metadata token with value 0x02000007 refers to row 7 in the TypeDef table in the current scope. Similarly, token 0x0400001A refers to row 26 (decimal) in the FieldDef table in the current scope. Row zero of a metadata table never contains data, so a metadata token whose RID is zero is referred to as a nil token. The metadata API defines a host of such nil tokens, one for each token type, such as mdTypeRefNil, with the value 0x01000000.
Note
The preceding explanation of RIDs is conceptual; in reality, the physical layout of the metadata is much more complicated. In addition, string tokens (mdString) are slightly different — the lower 3 bytes are not record identifiers, but represent an offset to the string's starting location in the metadata string pool.
Using Metadata Tokens
Each DefineXXX method in the metadata API returns a token that can be passed to a GetXXX method to obtain its associated attributes.
Metadata tokens are defined within a scope. For example, a metadata token with value N completely identifies, within a given scope, a record that contains details about a type definition. However, in a different scope, a metadata token with that same value N might specify a completely different record.
A metadata token is not an immutable metadata object identifier. When two scopes are merged, tokens from the imported scope are remapped into tokens in the emitted scope. When a metadata scope is saved, various format optimizations can result in token remapping.
Token Types
The following table lists the metadata token types, the abstraction that each token type represents, and the name of the metadata table that contains the abstraction's metadata. All token types are variations of mdToken, which is the basic token type.
Token Type |
Metadata Table |
Abstraction |
|---|---|---|
mdModule |
Module |
Module: A compilation unit, an executable, or some other development unit, deployment unit, or run-time unit. It is possible (though not required) to declare attributes on the module as a whole, including a name, a GUID, custom attributes, and so forth. |
mdModuleRef |
ModuleRef |
Module reference: A compile-time reference to a module, which records the source for type and member imports. |
mdTypeDef |
TypeDef |
Type declaration: Declaration of either a runtime reference type (class or interface) or a value type. |
mdTypeRef |
TypeRef |
Type reference: Reference to either a runtime reference type or a value type. In a sense, the collection of type references in a module is the collection of compile-time import dependencies. |
mdMethodDef |
MethodDef |
Method definition: Definition of a method as a member of a class or interface, or as a global module-level method. |
mdParamDef |
ParamDef |
Parameter declaration: Definition of an optional data structure that stores additional metadata for the parameter. It is not necessary to emit a data structure for each parameter in a method. However, when there is additional metadata to persist for the parameter, such as marshaling or type-mapping information, an optional parameter data structure can be created. |
mdFieldDef |
FieldDef |
Field declaration: Declaration of a variable as a data member of a class or interface, or declaration of a global, module-level variable. |
mdProperty |
Property |
Property declaration: Declaration of a property as a member of a class or interface. |
mdEvent |
Event |
Event declaration: Declaration of a named event as a member of a class or interface. |
mdMemberRef |
MemberRef |
Member reference: Reference to a method or field. A member reference is generated in metadata for every method invocation or field access that is made by any implementation in the current module, and a token is persisted in the Microsoft intermediate language (MSIL) stream. There is no runtime support for property or event references. |
mdIfaceImpl |
IfaceImpl |
Interface implementation: A specific class’s implementation of a specific interface. This metadata abstraction enables the storing of information that is the intersection of that which is specific to neither the class nor the interface. |
mdMethodImpl |
MethodImpl |
Method implementation: A specific class’s implementation of a method that is inherited using interface inheritance. This metadata abstraction enables information to be persisted that is specific to the implementation rather than to the contract. Method declaration information cannot be modified by the implementing class. |
mdCustomAttribute |
CustomAttribute |
Custom attribute: An arbitrary data structure associated with any metadata object that can be referenced with an mdToken. (An exception is that custom attributes themselves cannot have custom attributes.) |
mdPermission |
Permission |
Permission set: A declarative security permission set associated with mdTypeDef, mdMethodDef, and mdAssembly. For more information, see Adding Declarative Security Support. |
mdTypeSpec |
TypeSpec |
Type constructor: A method that obtains a token for a type (such as a boxed value type) that can be used as input to any MSIL instruction that takes a type. |
mdSignature |
Signature |
Stand-alone signature: A local variable signature in the portable executable (PE) file or a method signature that is passed to an MSIL instruction. |
mdString |
String |
User string: A string that is passed to an MSIL instruction. |
Note
The preceding list does not include two separate token types, one for a field reference and another for a method reference, as might be expected. Field and method references share the same table and can be referenced using the token type mdMemberRef.
Extensibility and Abstractions
Runtime metadata is extensible, which is important in the following scenarios:
To represent constraints or higher-level abstractions defined by the Common Language Specification (CLS). The CLS is a specification for conventions that languages and tools agree to support in a uniform way for better language integration. The CLS may constrain parts of the common type system model, and the CLS may introduce higher-level abstractions that are layered over the common type system. Metadata must be able to capture these types of development-time abstractions used by tools, even though the abstractions are not recognized or supported explicitly by the runtime.
To represent language-specific abstractions that are not part of the common type system and are not CLS abstractions. This enables languages like Visual C to not require separate header files or IDL files in order to use types, methods, and data members exported by compiled modules.
To encode in-member signature types and type modifiers that are used in language-specific overloading.
Metadata extensibility comes in the following forms:
Every metadata object can support custom attributes, and the metadata APIs provide a way to declare, enumerate, and retrieve custom attributes. Custom attributes can be identified by a type reference (mdTypeDef and mdTypeRef). The structure of the custom attribute is self-describing, using data members declared on the type, and the value encoding is browsable by any tool, including the runtime reflection services.
In addition to extending the common type system, it is possible to emit custom modifiers into member signatures. The runtime will honor these modifiers for method overloading and hiding as well as for binding, but will not enforce any language-specific semantics.