>>Microsoft Corporation

May 2009

[This documentation targets the Microsoft "Oslo" May 2009 CTP and is subject to change in future releases. Blank topics are included as placeholders.]

Sections:
1: Introduction to "M"
2: Lexical Structure
3: Text Pattern Expressions
4: Productions
5: Rules
6: Languages
7: Types
8: Computed and Stored Values
9: Expressions
10: Module
11: Attributes
12: Catalog
13: SQL Mapping
14: Glossary

7 Types

The types of the M language are divided into two main categories: intrinsic types and derived types. An intrinsic type is a type that cannot be defined using M language constructs but rather is defined entirely in the M Language Specification. An intrinsic type (e.g. Number, Entity, Collection) may name a super-type as part of its specification. Values are an instance of exactly one intrinsic type, and conform to the specification of that one intrinsic type and all of its super-types.

A derived type (e.g. Integer32, Person, Cars) is a type whose definition is constructed in M source text using the type constructors that are provided in the language. A derived type is defined as a constraint over another type, which creates an explicit subtyping relationship. Values conform to any number of derived types simply by virtue of satisfying the derived type’s constraint. There is no explicit affiliation between a value and a derived type – rather a given value that conforms to a derived type’s constraint may be interpreted as that type or any other derived type using type ascription.

>>>>7.1 Type declaration

M offers a broad range of options in defining types. Any expression that returns a collection can be declared as a type. The type predicates for entities and collections are expressions and fit this form. A type declaration may explicitly enumerate its members or be composed of other types.

The syntax for a type declaration follows:

TypeDeclaration:
type  Identifier  ; 
type  Identifier  InitializationExpression  ; (opt) 
type  Identifier  EntityTypeExpression  ; (opt)
type  Identifier  EntityTypeExpression  where  WhereExpressions  ;
type  Identifier  :  Expression  ; 
type  Identifier  :  TypeReferences  ; 
type  Identifier  :  TypeReferences EntityTypeExpression  ; (opt) 
type  Identifier  :  TypeReferences EntityTypeExpression  where WhereExpressions;

The Identifier in a type declaration introduces a new symbol into the module level scope. 

TypeReference:
QualifiedIdentifier 

TypeReferences:
TypeReference 
TypeReferences  ,  TypeReference 

The QualifiedIdentifier in TypeReference must either refer to a type declaration available within the current scope (§9.2.1).

The declaration:

type SomeNewType;

declares a new type SomeNewType with no constraints. Any value satisfies this type.

The following example explicitly enumerates the values of type PrimaryColors and uses it in the EntityExpression which defines the type Car.

type PrimaryColors {"Red", "Blue", "Yellow"}
type Car {
    Make : Text;
    Model : Text;
    Color : PrimaryColors;
}

These common cases do not require a colon between the declaration name and the definition. 

Type declarations can be built up from expressions that return collections. The type PrimaryColors above could be constructed from singleton sets.

type PrimaryColors2 : {"Red"} | {"Blue"} | {"Yellow"}

Since the expression {"Red"} | {"Blue"} | {"Yellow"} == {"Red", "Blue", "Yellow"} the two declarations are equivalent.

However an expression which does not return a collection is not a semantically valid type.

type NonSense : 1 + 1;

is a syntactically valid declaration, but not useful as a type since no value of X would ever satisfy, the following expression because 2 is not a collection.

X in 2

Entity types may be composed as well. Consider the following two distinct types:

type Vehicle {
    Owner : Text;
    Registration : Text;
}
type HasWheels {
    Wheels : Integer32;
}

The type Vehicle requires that instances have Owner and Registration fields. The type HasWheels requires instances have a Wheels field. These two types can be combined into a new type Car that requires Owner, Registration, and Wheels fields.

type Car : Vehicle & HasWheels;

In this usage, ampersand requires that Car meet all the requirements of both arguments. 

This definition of Car can be further restricted since cars have 4 wheels. Such restrictions can be specified with a constraint (§9.14.1).

type Car2 : Vehicle & HasWheels where value.Wheels == 4;

It is common to extend types with additional fields and restrict values. M provides the following syntax to simplify this case.

type Car3 : Vehicle {
    Wheels : Integer32;
} where value.Wheels == 4;

>>>>7.2 Subtyping

M is a structurally typed language rather than a nominally typed language like C++ or C#. A structural type is a specification for a set of values. Two types are equivalent if the exact same collection of values conforms to both regardless of the name of the types. 

It is not required that a type be named to be used. A type expression is allowed wherever a type reference is required. Types in M are simply expressions that return collections.

If every value that conforms to type A also conforms to type B, we say that A is a subtype of B (and that B is a super-type of A). Subtyping is transitive, that is, if A is a subtype of B and B is a subtype of C, then A is a subtype of C (and C is a super-type of A). Subtyping is reflexive, that is, A is a (vacuous) subtype of A (and A is a super-type of A). 

>>>>>7.3 Operators

Types are considered collections of all values that satisfy the type predicate. For that reason, any operation on a collection (§7.6.2) can be applied to a type and a type can be manipulated with expressions like any other collection value.

The relational operators ( <, >, <=, >=, ==, != ) compare the value spaces of two types and return a Logical value. For example, the operator <= on types computes the subtype relation. 

(Car <= Vehicle) == true

(Car <= HasWheels) == true

(Car <= Colors) == false

The where constraint restricts the value space of a type to those elements satisfying the right operand's logical expression.

The following binary operations take Collection as a left operand.

The union and intersection operators (|, &) operate on the type's value spaces. Intersection, &, can be thought of as specialization, restriction, or subtyping. Union, |, can be thought of as generalization or inducing a supertype.

The following postfix operators take types as a left operand.

? is a postfix operator that adds null to the value space of its operand. T? is equivalent to

T | { null }

The multiplicities lift a type to a collection of that type with the appropriate cardinality. For example:

Date*        // A collection of any number of dates

Person+      // A collection of one or more people

Wheel#2..4   // A collection of two to four wheels

>>>>7.4 Intrinsic Types

The following table lists the intrinsic types that are defined as part of the M Language Specification:

>>>>7.4.1 Any

All values are members of this type. 

The following binary operations take Any as a left operand.

The in operator returns true if some member of the right operand is equal (==) to the left operand.  The !in operator returns true if the in operator would return false.

>>>>7.4.2 General

All values that are not members of Entity or Collection (or null) are members of this type. It has no additional operators beyond those defined on Any.

>>>>>7.4.3 Number

Number is an abstract type with four subtypes enumerated below. Each of these subtypes is further refined to a type with a precision. A type of a smaller precision may always be converted to the same type of a larger precision. Converting from a larger precision to a smaller precision tests for overflow at runtime.

The arithmetic operations (+, -, *, /, %) defined above are specialized to return the most specific type of its operands (e.g. Integer8 + Integer8 returns Integer8, Decimal9 + Decimal38 returns Decimal38)

>>>>7.4.3.1 Operators

The following unary operations take Number as a right operand.

The following binary operations take Number as a left operand.

The following operations may cause underflow and overflow errors:

The predefined unary - operator

The predefined +, -, *, and / binary operators

Explicit numeric conversions from one Number type to another

The following operations may cause a divide-by-zero error:

The predefined / and % binary operators

>>>>7.4.3.2 AutoNumber

Unique numbers can be generated with the AutoNumber computed value. This is a special form for ensuring unique identities. Consider the following example:

type Person {
    Id : Integer32 => AutoNumber();
    Name : Text;
    Age : Integer32;
    Spouse : Person;
} where identity Id; People : Person*;

Each instance of Person will receive an Id value that is unique for each extent that contains Person instances. 

AutoNumber has a number of restrictions. The default value should not be overridden and AutoNumber may only be used on identity fields. 

>>>>7.4.4 Text

The representation of text is implementation dependent. 

The following postfix operator takes Text as a left operand.

The postfix # operator returns the count of characters in a Text string.

The following binary operations take Text as a left operand.

The binary + operator concatenates two Text strings.

The relational operators perform a lexicographic comparison on the Text strings and return a Logical value.

>>>>7.4.4.1 Members

The following members are defined on Text.

Count() : Unsigned;

Like(pattern : Text) : Logical;

PatternIndex(pattern : Text) : Integer;

Count provides the number of characters in the text. 

Like returns true if the input is matched by the pattern. 

PatternIndex returns the starting position of the pattern in the text or -1 if the pattern is not found.

The pattern is of the following form:

Pattern
PatternElement
Pattern PatternElement

PatternElement
NormalCharacter
-
%
[  NormalCharacter - NormalCharacter ]
[^ NormalCharacter - NormalCharacter ]

Dash matches any single character. Percent matches zero or more characters. A character range matches any single character in the range. And an excluded character range matches any character not in the range.

>>>>7.4.4.2 Declaration

The # qualifier is overloaded on Text declarations to constrain the length of the text field. The expression

Text#N;

is equivalent to

Text where value.Count == N;

This special form is specific to the Text type.

>>>>>>7.4.5 Logical

The following unary operator takes Logical as an operand.

The following binary operations take Logical as a left operand.

The following ternary operator takes Logical as a right operand.

>>>>>7.4.6 Binary

Binary defines one member Count which returns the number of bytes in the binary value. The following expression returns true.

0x3333.Count == 2

The following unary operator takes Binary as a right operand.

The ~ operator computes the bitwise negation of its operand.

The following binary operations take Binary as a left operand.

The bitwise and (&), bitwise exclusive or (^), and bitwise or (|) operators implicitly convert their operands to the same length. The smaller operand is padded with zeros on the left.

The precedence of the bitwise and, or, and exclusive or is lower than it is in many other languages. 

>>>>7.4.7 Guid

The following binary operations take Guid as a left operand.

Guids are created with the system defined NewGuid computed value.

NewGuid() : Guid;

>>>>>7.4.8 Date

The following binary operations take Date as a left operand:

>>>>>>>7.4.9 DateTime

The following binary operations take DateTime as a left operand:

>>7.4.10 DateTimeOffset

The following binary operations take DateTime as a left operand:

>>7.4.11 Time

The following binary operations take Time as a left operand.

>>>>>7.5 Entity

An EntityTypeExpression specifies the members for a set of entity values (commonly referred to as entities). Those members can be either fields or computed values.

Entity types are distinct from extents. The definition of an entity type does not imply allocation of storage. Storage is allocated when an extent of entity type is declared within a module. 

The fields of an entity can be assigned default values and the values can be constrained with expressions. The names of all fields must be distinct.

>>>>>7.5.1 Declaration

The following syntax defines a collection of all possible instances that satisfy the structure and constraint.

EntityTypeExpression:
{  EntityMemberDeclarations  } 

EntityMemberDeclarations:
EntityMemberDeclaration 
EntityMemberDeclarations  EntityMemberDeclaration 

EntityMemberDeclaration:
FieldDeclaration 

Entity declarations share FieldDeclaration with module. An entity type which through intersection or refinement results in two default values for the same named field is an error. 

>>>>>7.5.2 Identity

The identity constraint controls the representation of identity. If it is specified the selected fields are used to represent the identity. If no identity constraint is specified, the entity cannot be referenced or compared. Placing the identity constraint on a field makes that field initialize only. It cannot be updated.

The identity constraint may be specified either on entity declarations or on extent declarations. The identity constraint requires that the elements in the constraint are unique within each extent (not across extents) as with the unique constraint. An identity declaration on a derived type supersedes that of any types it derives from. As a result, there can be only one identity constraint on an entity or an extent. 

Consider the following example:

type Container {
    Id : Integer32;
    Capacity: Integer32;
} where identity Id;

CoffeeCups : Container* { {Id => 1, Capacity => 12} }
WaterBottles : Container* { {Id => 1, Capacity => 12} }

EqualityTest() {
    from c in CoffeeCups
    from w in WaterBottles
    where c == w
    select "Never"
}

It is legal for the two extents to contain instances whose Id fields are equal. Having the same Id field does not equate the instances. The computed value EqualityTest will always return the empty collection because identity is relative to an extent.

An implementation of M may restrict the types of fields used to form identity. 

>>>7.5.3 Operators

The following binary operations take Entity as a left operand.

The equality operations on entities compare identity (shallow equal).  == returns true if both operands refer to an instance with the same identity in the same collection.

>>>>>>>>>7.5.4 Members

The following member is defined on all entities:

FieldNames() : Text*;

FieldNames returns the string names of each label in an instance. This member is not affected by ascription and does not return names of computed values or missing default values.

>>>>>7.5.6 Indexer

Entities have a default indexer that accepts field name as text and returns the value of the field if present or null.  

{Name = "Bob"}("Name") == "Bob"

{Name = "Bob"}("Age")  == null

The indexer accesses the underlying instance data without interpretation by the type. This allows the indexer to access field values that may be hidden by a computed value. Consider:

type Hider {

    Name() : Text { "Hides instance values" };

}

Given the above declaration, the following two expressions would evaluate to true.

({Name = "Underlying value"} : Hider).Name == "Hides instance values"

({Name = "Underlying value"} : Hider)("Name") == "Underlying value"

>>7.5.6 Ascription

An entity defines a constraint over a set of values. An entity type can be ascribed to any value which satisfies its constraint. Ascribing an entity type allows the computed values defined in the entity to be applied to the value.

Consider the following two entities and two instances (the square root (SQRT) and absolute value (ABS) functions must be provided by a library, they are not intrinsic).

type PointOnPlane {
    X : Single;
    Y : Single;
    DistanceFromOrigin : Single { SQRT(X * X + Y * Y) }
}

type PointOnLine {
    X : Single;
    Y : Single;
    DistanceFromOrigin : Single { ABS(X) * SQRT(2) }
} where X == Y;

Point1 => {X => 1, Y => 1};
Point2 => {X => 0, Y => 1};

Both entities define fields X and Y and a computed value DistanceFromOrigin although the implementation of the computed value differs. The first entity, PointOnPlane, allows any X,Y combination—the entire X,Y plane. The second entity, PointOnLine, has a constraint that restricts the values that can be members of the type.

Point1 is a member of both PointOnPlane and PointOnLine. Both declarations of DistanceFromOrigin are valid and yield the same result

Point2 can be ascribed PointOnPlane, but not of PointOnLine since the constraint X == Y is not satisfied. This prevents the alternative declaration of DistanceFromOrigin from producing an incorrect result. 

>>7.5.7 Constructor

A ComputedValueDeclaration with the same name as an entity type declaration is a constructor rather than a member. The formal parameters of a constructor are field names of the entity. Actual parameters are bound to the corresponding fields in a new entity instance. A constructor declaration need not specify a body. 

Consider the following example:

type Person {
    Name : Text;
    Age : Integer32;
    Person(Name,Age);
}
People : Person*
{
    Person("John", 23),
    Person("Mary", 22)
}

The extent People will contain two elements with Name fields equal to "John" and "Mary".

>>>>7.6 Collections

Collections are unordered and may contain elements which are equal. M provides operators to construct strongly typed collections and in some cases defined below escalates members on elements to members on the collection.

>>>>7.6.1 Declaration

New collection types are defined by a type constructor and a multiplicity ( +, *, #m..n ).

>>>>The default value for a collection type is the empty collection, written {}. The one-to-many multiplicity constraint forbids an empty collection, so must have at least one member on initialization.

>>>>>7.6.2 Operators

The following postfix unary operator takes Collection as a left operand.

The following binary operations take Collection as a left operand.

>>>For the operators that return a collection, the inferred element type of the resulting collection is the most specific type which the elements of both operands may be converted to.

>>>7.6.3 Members

The following members are defined on all collections:

Choose() : Any;

Count() : Unsigned;       

Distinct() : Collection;    

Choose picks an arbitrary element from a collection. The return type is the element type. The result of calling Choose on an empty collection is undefined.

Count returns the total number of elements in a collection. The return type is a Number.

Distinct removes all duplicates in a collection. The return type is the same as the collection.

The following members are defined on collections of type Logical*:

All() : Logical;                     

Exists() : Logical;

All returns false if false is an element of the collection and true otherwise. Exists returns true if true is an element of the collection and false otherwise.

The following members are defined on collections that are subtypes of Number*:

Average() : Scientific;

Maximum() : Number;

Minimum() : Number;

Sum() : Number;

Maximum, Minimum, and Sum are specialized to return the element type of the collection.

Average computes the Sum of the collection and divides that by the Count.

Maximum returns the largest value in the collection.

Minimum returns the smallest value in the collection.

Sum returns the arithmetic summation of the values in the collection.

>>>>7.6.4 Indexers

A collection may be accessed using language generated indexers of two kinds, selectors and projectors. A selector extracts members of a collection with a member that matches a value. A projector extracts all values of a field from a collection. Both of these operations can be accomplished with query expressions; however, this notation is more compact.

>>>>7.6.4.1 Selectors

The compiler will generate indexers for all fields of Person for Person*.

Consider the following example:

type Person {
    Id : Integer64 => AutoNumber();
    Name : Text;
    HairColor : Text;
} where identity Id, unique Name;

People : Person* {
    {Name => "Mary", HairColor => "Brown"},
    {Name => "John", HairColor => "Brown"},
    {Name => "Fritz", HairColor => "Blue"}
};

 Consider the following expressions:

People.Name("Mary")

evaluates to:

{{Name => "Mary", HairColor => "Brown" }}

People.Name("Bill")

evaluates to:

{}

People.HairColor("Brown")

evaluates to:

{

    {Name => "Mary", HairColor => "Brown"},

    {Name => "John", HairColor => "Brown"}

}

// Assuming the Fritz record was assigned the Id 123

People.Id(123)

evaluates to:

{{Name => "Fritz", HairColor => "Blue"}}

The expression:

Collection.MemberField(Expression)

is equivalent to:

from c in Collection

where c.MemberField == Expression

select c

The identity auto indexer is special in that it is also an indexer directly on the collection, so the following expression is legal:

People(123) == {{Name => "Fritz", HairColor => "Blue"}}

If the designer chose a different representation for identity, it would be the default indexer as shown in the following variant of the above example:

type Person {
    Name : Text;
    HairColor : Text;
} where identity Name;

People("Mary") == {{Name => "Mary", HairColor => "Brown" }}

Assuming the identity constraint for a collection is defined using the following pattern:

identity(IdField1, IdField2, ...)

the following expression

Collection (Expression1, Expression2, ...)

is equivalent to:

(from c in Collection

where c.IdentityField1 == Expression1, c.IdField2 == Expression2, ...

select c).Choose

>>>>7.6.4.2 Projectors

Projectors return the values 0f one field from each member of a collection. 

Again, consider the following example:

type Person {
    Id : Integer64 => AutoNumber();
    Name : Text;
    HairColor : Text;
} where identity Id, unique Name;

People : Person* {
    {Name => "Mary", HairColor => "Brown"},
    {Name => "John", HairColor => "Brown"},
    {Name => "Fritz", HairColor => "Blue"}
}

The following expressions all evaluate to true:

People.Name == {"Mary", "John", "Fritz"}

People.HairColor == {"Brown", "Brown", "Blue"}

People.HairColor.Distinct == {"Brown", "Blue"}

Note that the returned collection may have duplicates. To obtain a duplicate free collection, use Distinct.

The expression:

Collection.MemberField

is equivalent to:

from c in Collection
select c.MemberField

In the event that the identifier for the projector is equal to a member on collection, the projector is not added. Specifically, Choose, Count, and Distinct will not be added as projectors.

>>>>7.6.5 Uniqueness

Collections in M may contain multiple copies of the same element. The constraint unique value limits the number of elements in a collection to 1. No two elements in the collection will return true for ==.

The unique constraint may also take an expression or a comma separated list of expressions. In this case, the constraint will ensure no two elements are equal on every expression in the list.

>>>>7.7 Null

Null is a type with a single value null. It is used in conjunction with other types to add null to the value space and make a nullable type. Nullable types can be specified with the postfix operator ? or with a union of the type and Null.

The type below has two nullable fields, SSN and Spouse. 

type Person {
    Name : Text;
    SSN : Text?;
    Spouse : Person | Null;
}

Nullability is idempotent. T?? is the same as T? Collections cannot be made nullable therefore T*? is not a legal type. Elements of collections can be nullable so T?* is a legal type. 

Except as noted binary operations defined to take a left operand of T, right operand of S and return type of R are lifted to accept T?, S? and return R?. If either actual operand is null, the operation will return null.  Logical operations && and || are not lifted.

The following binary operations take Null as a left operand.

The return type of ?? is specialized to the type for the left operand without the null value. The type of the right operand must be compatible with the type of the left operand.

The default value of type Null is null.