November 2009

[This documentation targets the Microsoft SQL Server Modeling CTP (November 2009) and is subject to change in future releases. Blank topics are included as placeholders.]

Sections:
1: Introduction to "M"
2: Lexical Structure
3: Text Pattern Expressions
4: Productions
5: Rules
6: Languages
7: Types
8: Computed and Stored Values
9: Expressions
10: Module
11: Attributes
12: Catalog
13: Standard Library
14: Glossary

7 Types

The types of the M language are divided into two main categories: intrinsic types and derived types. An intrinsic type is a type that cannot be defined using M language constructs but rather is defined entirely in the M Language Specification. An intrinsic type (e.g. Number, Entity, Collection) may name a super-type as part of its specification. Values are an instance of exactly one intrinsic type, and conform to the specification of that one intrinsic type and all of its super-types.

A derived type (e.g. Integer32, Person, Cars) is a type whose definition is constructed in M source text using the type constructors that are provided in the language. A derived type is defined as a constraint over another type, which creates an explicit subtyping relationship. Values conform to any number of derived types simply by virtue of satisfying the derived type’s constraint. There is no explicit affiliation between a value and a derived type – rather a given value that conforms to a derived type’s constraint may be interpreted as that type or any other derived type using type ascription.

7.1 Type declaration

M offers a broad range of options in defining types. Any expression that returns a collection can be declared as a type. The type predicates for entities and collections are expressions and fit this form. A type declaration may explicitly enumerate its members or be composed of other types.

The syntax for a type declaration follows:

syntax TypeDeclaration

    = "type" Identifier ";" 

    | "type" Identifier InitializationExpression ";"? 

    | "type" Identifier EntityTypeExpression ";"?

    | "type" Identifier EntityTypeExpression "where" 

      WhereExpressions ";"

    | "type" Identifier ":" Expression ";" 

    | "type" Identifier ":" TypeReferences ";" 

    | "type" Identifier ":" TypeReferences EntityTypeExpression ";"? 

    | "type" Identifier ":" TypeReferences EntityTypeExpression "where"

      WhereExpressions ";";

The Identifier in a type declaration introduces a new symbol into the module level scope. 

syntax TypeReference

    = QualifiedIdentifier;

syntax TypeReferences

    = TypeReference 

    | TypeReferences "," TypeReference;

The QualifiedIdentifier in TypeReference must either refer to a type declaration available within the current scope (§9.2.1).

The declaration:

type SomeNewType;

declares a new type SomeNewType with no constraints. Any value satisfies this type.

The following example explicitly enumerates the values of type PrimaryColors and uses it in the EntityExpression which defines the type Car.

type PrimaryColors {"Red", "Blue", "Yellow"}

type Car {

    Make : Text;

    Model : Text;

    Color : PrimaryColors;

}

These common cases do not require a colon between the declaration name and the definition. 

Type declarations can be built up from expressions that return collections. The type PrimaryColors above could be constructed from singleton sets.

type PrimaryColors2 : {"Red"} | {"Blue"} | {"Yellow"}

Since the expression {"Red"} | {"Blue"} | {"Yellow"} == {"Red", "Blue", "Yellow"} the two declarations are equivalent.

However an expression which does not return a collection is not a semantically valid type.

type NonSense : 1 + 1;

is a syntactically valid declaration, but not useful as a type since no value of X would ever satisfy, the following expression because 2 is not a collection.

X in 2

Entity types may be composed as well. Consider the following two distinct types:

type Vehicle {

    Owner : Text;

    Registration : Text;

}

type HasWheels {

    Wheels : Integer32;

}

The type Vehicle requires that instances have Owner and Registration fields. The type HasWheels requires instances have a Wheels field. These two types can be combined into a new type Car that requires Owner, Registration, and Wheels fields.

type Car : Vehicle & HasWheels;

In this usage, ampersand requires that Car meet all the requirements of both arguments. 

This definition of Car can be further restricted since cars have 4 wheels. Such restrictions can be specified with a constraint (§9.14.1).

type Car2 : Vehicle & HasWheels where value.Wheels == 4;

It is common to extend types with additional fields and restrict values. M provides the following syntax to simplify this case.

type Car3 : Vehicle {

    Wheels : Integer32;

} where value.Wheels == 4;

7.2 Subtyping

M is a structurally typed language rather than a nominally typed language like C++ or C#. A structural type is a specification for a set of values. Two types are equivalent if the exact same collection of values conforms to both regardless of the name of the types. 

It is not required that a type be named to be used. A type expression is allowed wherever a type reference is required. Types in M are simply expressions that return collections.

If every value that conforms to type A also conforms to type B, we say that A is a subtype of B (and that B is a super-type of A). Subtyping is transitive, that is, if A is a subtype of B and B is a subtype of C, then A is a subtype of C (and C is a super-type of A). Subtyping is reflexive, that is, A is a (vacuous) subtype of A (and A is a super-type of A). 

7.3 Operators

Types are considered collections of all values that satisfy the type predicate. For that reason, any operation on a collection (§7.6.2) can be applied to a type and a type can be manipulated with expressions like any other collection value.

The relational operators ( <, >, <=, >=, ==, != ) compare the value spaces of two types and return a Logical value. For example, the operator <= on types computes the subtype relation. 

(Car <= Vehicle) == true

(Car <= HasWheels) == true

(Car <= Colors) == false

The where constraint restricts the value space of a type to those elements satisfying the right operand's logical expression.

The following binary operations take Collection as a left operand.

The union and intersection operators (|, &) operate on the type's value spaces. Intersection, &, can be thought of as specialization, restriction, or subtyping. Union, |, can be thought of as generalization or inducing a supertype.

The following postfix operators take types as a left operand.

? is a postfix operator that adds null to the value space of its operand. T? is equivalent to

T | { null }

The multiplicities lift a type to a collection of that type with the appropriate cardinality. For example:

{Date*}        // A collection of any number of dates

{Person+}      // A collection of one or more people

{Wheel#2..4}   // A collection of two to four wheels

7.4 Intrinsic Types

The following table lists the intrinsic types that are defined as part of the M Language Specification:

7.4.1 Any

All values are members of this type. 

The following binary operations take Any as a left operand.

The in operator returns true if some member of the right operand is equal (==) to the left operand. The !in operator returns true if the in operator would return false.

7.4.2 General

All values that are not members of Entity or Collection (or null) are members of this type. It has no additional operators beyond those defined on Any.

7.4.2.1 Members

General types have one member ToText which returns a default text representation of the value. 

ToText() : Text;

Examples:

2.ToText

"Hello".ToText

True.ToText

7.4.3 Number

Number is an abstract type with four subtypes enumerated below. Each of these subtypes is further refined to a type with a precision. A type of a smaller precision may always be converted to the same type of a larger precision. Converting from a larger precision to a smaller precision tests for overflow at runtime.

The arithmetic operations (+, -, *, /, %) defined below are specialized to return the most specific type of its operands (e.g. Integer8 + Integer8 returns Integer8, Decimal9 + Decimal38 returns Decimal38)

7.4.3.1 Operators

The following unary operations take Number as a right operand.

The - operator computes the numeric negation of its operand.

The ! operator computes the bitwise negation of its operand.

The following binary operations take Number as a left operand.

The following operations may cause underflow and overflow errors:

• The predefined unary - operator
• The predefined +, -, *, and / binary operators
• Explicit numeric conversions from one Number type to another

The following operations may cause a divide-by-zero error:

• The predefined / and % binary operators

The bitwise and (&), bitwise exclusive or (^), and bitwise or (|) operators implicitly convert their operands to the same length. The smaller operand is padded with zeros on the left.

The precedence of the bitwise and, or, and exclusive or is lower than it is in many other languages. 

7.4.3.2 AutoNumber

Unique numbers can be generated with the AutoNumber computed value. This is a special form for ensuring unique identities. Consider the following example:

type Person {

    Id : Integer32 => AutoNumber();

    Name : Text;

    Age : Integer32;

    Spouse : Person;

} where identity Id;

People : {Person*};

Each instance of Person will receive an Id value that is unique for each extent that contains Person instances. 

AutoNumber has a number of restrictions. The default value should not be overridden and AutoNumber may only be used on identity fields. 

7.4.4 Text

Text is a sequence of Unicode characters.

The following postfix operator takes Text as a left operand.

The postfix # operator returns the count of characters in a Text string.

The following binary operations take Text as a left operand.

The binary + operator concatenates two Text strings.

The relational operators perform a lexicographic comparison on the Text strings and return a Logical value.

7.4.4.1 Members

The following members are defined on Text.

Count() : Unsigned;

Like(pattern : Text) : Logical;

PatternIndex(pattern : Text) : Integer;

Count provides the number of characters in the text. 

Like returns true if the input is matched by the pattern. 

PatternIndex returns the starting position of the first match of the pattern in the text or -1 if the pattern is not found.

The pattern is of the following form:

syntax Pattern

    = PatternElement

    | Pattern PatternElement;

syntax PatternElement

    = NormalCharacter

    | "_"

    | "%"

    | "["  NormalCharacter - NormalCharacter "]"

    | "[^" NormalCharacter - NormalCharacter "]"

This will be aligned with grammars.

Dash matches any single character. Percent matches zero or more characters. A character range matches any single character in the range. And an excluded character range matches any character not in the range.

7.4.4.2 Length declaration

Text declarations may take a single parameter to specify the maximum length of the text field. The expression

Text(N);

is equivalent to

Text where value.Count <= N;

The expression

Text(N,M)

is equivalent to

Text where value.Count >= N && value.Count <= M;

The expression

Text(N,null)

is equivalent to

Text where value.Count >= N;

This special form is specific to the Text type.

7.4.5 Logical

The following unary operator takes Logical as an operand.

The following binary operations take Logical as a left operand.

The following ternary operator takes Logical as a right operand.

7.4.6 Guid

The following binary operations take Guid as a left operand.

Guids are created with the system defined NewGuid computed value.

NewGuid() : Guid;

7.4.7 Date

The following binary operations take Date as a left operand:

7.4.8 DateTime

The following binary operations take DateTime as a left operand:

7.4.9 DateTimeOffset

The following binary operations take DateTimeOffset as a left operand:

7.4.10 Time

The following binary operations take Time as a left operand.

7.5 Entity

An EntityTypeExpression specifies the members for a set of entity values (commonly referred to as entities). Those members are fields.

Entity types are distinct from extents. The definition of an entity type does not imply allocation of storage. Storage is allocated when an extent of entity type is declared within a module. 

The fields of an entity can be assigned default values and the values can be constrained with expressions. The names of all fields within an entity type declaration must be distinct.

7.5.1 Declaration

The following syntax defines a collection of all possible instances that satisfy the structure and constraint.

syntax EntityTypeExpression

    = "{"  EntityMemberDeclarations "}";

syntax EntityMemberDeclarations

    = EntityMemberDeclaration 

    | EntityMemberDeclarations  EntityMemberDeclaration; 

syntax EntityMemberDeclaration

    = FieldDeclaration; 

An entity type which through intersection or refinement results in two default values for the same named field is an error. 

7.5.2 Identity

The identity constraint controls the representation of identity. If it is specified the selected fields are used to represent the identity. If no identity constraint is specified, the entity cannot be referenced or compared.

The identity constraint may be specified either on entity declarations or on extent declarations. The identity constraint requires that the elements in the constraint are unique within each extent (not across extents) as with the unique constraint. An identity declaration on a derived type supersedes that of any types it derives from. As a result, there can be only one identity constraint on an entity or an extent. 

Consider the following example:

type Container {

    Id : Integer32;

    Capacity: Integer32;

} where identity Id;

CoffeeCups : {Container*} { {Id => 1, Capacity => 12} }

WaterBottles : {Container*} { {Id => 1, Capacity => 12} }

EqualityTest() {

    from c in CoffeeCups

    from w in WaterBottles

    where c == w

    select "Never"

}

It is legal for the two extents to contain instances whose Id fields are equal. Having the same Id field does not equate the instances. The computed value EqualityTest will always return the empty collection because identity is relative to an extent.

An implementation of M may restrict the types of fields used to form identity. 

7.5.3 Operators

The following binary operations take Entity as a left operand.

The equality operations on entities compare identity (shallow equal).  == returns true if both operands refer to an instance with the same identity in the same collection.

7.5.4 Ascription

An entity defines a constraint over a set of values. An entity type can be ascribed to any value which satisfies its constraint. Ascribing an entity type allows the computed values defined in the entity to be applied to the value.

Consider the following two entities and two instances (the square root (SQRT) and absolute value (ABS) functions must be provided by a library, they are not intrinsic).

type PointOnPlane {

    X : Single;

    Y : Single;

    DistanceFromOrigin : Single { SQRT(X * X + Y * Y) }

}

type PointOnLine {

    X : Single;

    Y : Single;

    DistanceFromOrigin : Single { ABS(X) * SQRT(2) }

} where X == Y;

Point1 => {X => 1, Y => 1};

Point2 => {X => 0, Y => 1};

Both entities define fields X and Y and a computed value DistanceFromOrigin although the implementation of the computed value differs. The first entity, PointOnPlane, allows any X,Y combination—the entire X,Y plane. The second entity, PointOnLine, has a constraint that restricts the values that can be members of the type.

Point1 is a member of both PointOnPlane and PointOnLine. Both declarations of DistanceFromOrigin are valid and yield the same result.

Point2 can be ascribed PointOnPlane, but not of PointOnLine since the constraint X == Y is not satisfied. This prevents the alternative declaration of DistanceFromOrigin from producing an incorrect result. 

7.6 Collections

Collections are unordered and may contain elements which are equal. M provides operators to construct strongly typed collections and in some cases defined below escalates members on elements to members on the collection.

7.6.1 Declaration

New collection types are defined by a type constructor and a multiplicity ( +, *, #m..n ).

syntax CollectionTypeExpression

    = "{" UnaryExpression  "+" "}"

    | "{" UnaryExpression  "*" "}" 

    | "{" UnaryExpression "#"  IntegralRange "}";

syntax IntegralRange

    = IntegerLiteral 

    | IntegerLiteral  ".." 

    | IntegerLiteral  ".."  IntegerLiteral; 

The default value for a collection type is the empty collection, written {}. The one-to-many multiplicity constraint forbids an empty collection, so must have at least one member on initialization.

7.6.2 Operators

The following postfix unary operator takes Collection as a left operand.

The # operator returns the count of elements in the collection.

The following binary operations take Collection as a left operand.

The relational operator >= returns true if the left operand has every element of the right operand with equal or greater multiplicity. The remainder of the operators are defined as follows:

A == B ~ A >= B && B >= A

A > B  ~ A >= B && A != B

A != B ~ !(A == B)

A <= B ~ B >= A

A < B  ~ B > A

The where operator returns a new collection containing only the elements from the left operand that satisfy the predicate on the right when evaluated on the iteration variable value. If the type of the left operand is {T*} the type of the result will be {T*}.

The select operator returns a new collection containing equal number of elements as the left operand that are the result of evaluating the expression on the right over the iteration variable value. If the type of the left operand is {T*} and the result of evaluating the expression on the right is R, then the type of the result is {R*}.

The intersect and union ( &, | ) convert their operands to sets and perform set intersection, and union respectively.

For the operators that return a collection, the inferred element type of the resulting collection is the most specific type which the elements of both operands may be converted to.

7.6.3 Members

The following members are defined on all collections:

Choose() : Any;     

Count() : Unsigned; 

Distinct() : Collection;  

Choose picks an arbitrary element from a collection. The return type is the element type. The result of calling Choose on an empty collection is undefined.

Count returns the total number of elements in a collection. The return type is a Number.

Distinct removes all duplicates in a collection. The return type is the same as the collection.

The following members are defined on collections of type {Logical*}:

All() : Logical;          

Exists() : Logical;

All returns false if false is an element of the collection and true otherwise. Exists returns true if true is an element of the collection and false otherwise.

The following members are defined on collections that are subtypes of {Number*}:

Average() : Scientific;

Maximum() : Number;

Minimum() : Number;

Sum() : Number;

Maximum, Minimum, and Sum are specialized to return the element type of the collection.

Average computes the Sum of the collection and divides that by the Count.

Maximum returns the largest value in the collection.

Minimum returns the smallest value in the collection.

Sum returns the arithmetic summation of the values in the collection.

7.6.4 Indexers

A collection may be accessed using language generated indexers of two kinds, selectors and projectors. A selector extracts members of a collection with a member that matches a value. A projector extracts all values of a field from a collection. Both of these operations can be accomplished with query expressions; however, this notation is more compact.

7.6.4.1 Selectors

The compiler will generate indexers for all fields of Person for {Person*}.

Consider the following example:

type Person {

    Id : Integer64 => AutoNumber();

    Name : Text;

    HairColor : Text;

} where identity Id, unique Name;

People : {Person*} {

    {Name => "Mary", HairColor => "Brown"},

    {Name => "John", HairColor => "Brown"},

    {Name => "Fritz", HairColor => "Blue"}

};

 Consider the following expressions:

People.Name("Mary")

evaluates to:

{{Name => "Mary", HairColor => "Brown" }}

People.Name("Bill")

evaluates to:

{}

People.HairColor("Brown")

evaluates to:

{

    {Name => "Mary", HairColor => "Brown"},

    {Name => "John", HairColor => "Brown"}

}

// Assuming the Fritz record was assigned the Id 123

People.Id(123)

evaluates to:

{{Name => "Fritz", HairColor => "Blue"}}

The expression:

Collection.MemberField(Expression)

is equivalent to:

from c in Collection

where c.MemberField == Expression

select c

The identity auto indexer is special in that it is also an indexer directly on the collection, so the following expression is legal:

People(123) == {{Name => "Fritz", HairColor => "Blue"}}

If the designer chose a different representation for identity, it would be the default indexer as shown in the following variant of the above example:

type Person {

    Name : Text;

    HairColor : Text;

} where identity Name;

People("Mary") == {{Name => "Mary", HairColor => "Brown" }}

Assuming the identity constraint for a collection is defined using the following pattern:

identity(IdField1, IdField2, ...)

the following expression

Collection (Expression1, Expression2, ...)

is equivalent to:

(from c in Collection

where c.IdentityField1 == Expression1, c.IdField2 == Expression2, ...

select c).Choose

which always returns at most 1 element in the collection.

7.6.4.2 Projectors

Projectors return the values 0f one field from each member of a collection. 

Again, consider the following example:

type Person {

    Id : Integer64 => AutoNumber();

    Name : Text;

    HairColor : Text;

} where identity Id, unique Name;

People : {Person*} {

    {Name => "Mary", HairColor => "Brown"},

    {Name => "John", HairColor => "Brown"},

    {Name => "Fritz", HairColor => "Blue"}

}

The following expressions all evaluate to true:

People.Name == {"Mary", "John", "Fritz"}

People.HairColor == {"Brown", "Brown", "Blue"}

People.HairColor.Distinct == {"Brown", "Blue"}

Note that the returned collection may have duplicates. To obtain a duplicate free collection, use Distinct.

The expression:

Collection.MemberField

is equivalent to:

from c in Collection

select c.MemberField

In the event that the identifier for the projector is equal to a member on collection, the projector is not added. Specifically, Choose, Count, and Distinct will not be added as projectors.

7.7 Lists

Lists are ordered collections. 

7.7.1 Declaration

New collection types are defined by a type constructor and a multiplicity ( +, *, #m..n ).

syntax CollectionTypeExpression

    = "[" UnaryExpression "+" "]"

    | "[" UnaryExpression "*" "]" 

    | "[" UnaryExpression "#" IntegralRange "]";

Type Expression      Multiplicity

[TypeReference*]     Zero-to-Many

[TypeReference+]     One-to-Many

[TypeReference#N]    Exactly N

[TypeReference#Low..High]  From the Low bound to High bound

[TypeReference#Low..]      From the Low bound to any number

The default value for a collection type is the empty collection, written {}. The one-to-many multiplicity constraint forbids an empty collection, so must have at least one member on initialization.

7.7.2 Operators

The following postfix unary operator takes List as a left operand.

The # operator returns the count of elements in the list

The following binary operations take Collection as a left operand.

The >= operator returns true if the right operand has an equal or greater number of elements than the left operand and the element at every position in the left operand is less than or equal to the element in the same position in the right operand. The remaining relational operators are defined using the axioms defined for collections.

This is a lexicographic ordering to make text work. It does not align with collection. e.g.

[4,5,6,7] >= [1,2,3]

The where and select operators on lists behave similarly to collections; however, the return type is a list rather than a collection and order is maintained.

For the operators that return a collection, the inferred element type of the resulting collection is the most specific type which the elements of both operands may be converted to.

7.8 Null

Null is a type with a single value null. It is used in conjunction with other types to add null to the value space and make a nullable type. Nullable types can be specified with the postfix operator ? or with a union of the type and Null.

The type below has two nullable fields, SSN and Spouse. 

type Person {

    Name : Text;

    SSN : Text?;

    Spouse : Person | Null;

}

Nullability is idempotent. T?? is the same as T? Collections cannot be made nullable therefore {T*}? is not a legal type. Elements of collections can be nullable so {T? *} is a legal type. 

Except as noted, binary operations defined to take a left operand of T, right operand of S and return type of R are lifted to accept T?, S? and return R?. If either actual operand is null, the operation will return null. Logical operations && and || are not lifted.

The following binary operations take Null as a left operand.

The return type of ?? is specialized to the type for the left operand without the null value. The type of the right operand must be compatible with the type of the left operand.

The default value of type Null is null.