A DSL for Movies (Video)

Article
09/22/2010

[This content is no longer valid. For the latest information on "M", "Quadrant", SQL Server Modeling Services, and the Repository, see the Model Citizen blog.]

In this tutorial, you learn how to use the M.Grammar language (Mg) to write your own language for a Movies application. See the PDC Talk “Oslo”: Building Textual DSLs.

The scenario involves writing syntax and token rules to define a domain-specific language (DSL) specific to Movies. Next you use the “M” compiler to create a compiled version of this grammar. Finally, you input the compiled grammar plus some movie data into an Oslo tool which generates a parser, and transforms the movie data into an “M” file.

The tutorial illustrates M.Grammar features like:

Tokens with patterns.
Syntax rules with patterns and grammar terms.
The interleave keyword.
Output Projections.

Open a Basic DSL

In this task, you open “Intellipad” in Dynamic parser mode. You open an initial Mg language definition, and incrementally extend the definition.

Open Intellipad either from the start menu or by entering the following command (from c:\Program Files\Microsoft Oslo SDK 1.0\Bin):
```
"c:\program files\Microsoft Oslo SDK 1.0\bin\Intellipad\ipad.exe" /c:ipad-vs-samples.xaml
```
Note

Ipad-vs-samples.xaml is a configuration file that enables additional functionality, including the dynamic parse editing mode that we are about to use.

Copy the following code and paste it into the “Intellipad” window. Save it with the file name MoviePlayer.mg. Note that “Intellipad” is now in DSL Grammar Mode.

By default this code already includes some initial token, syntax, and interleave statements, and a Main syntax with a projection.

module MoviePlayer
{
    language MovieLanguage 
    {
        interleave Skippable 
            = Whitespace;

        syntax Main 
            = movies:Movie+ 
            => Movies{ valuesof(movies) };

        syntax Movie 
            = MovieStart name:MovieName 
            => name;

        syntax MovieName 
            = name:Name => name
            | name:NameVerbatim => name;

        syntax NameVerbatim 
            = '"' name:NameWithWhitespace '"' => name;
        token AlphaNumerical 
            = 'a'..'z' | 'A'..'Z' | '0'..'9';

        final token MovieStart 
            = "MoviePlayer";

        token Name 
            = AlphaNumerical+;

        token NameWithWhitespace 
            = (AlphaNumerical | Whitespace)+;

        token Whitespace
            = '\r' 
            | '\n' 
            | '\t' 
            | ' ';
    }
}

From the DSL menu item, select Tree Preview. In the Open dialog box, right-click the file pane and select New -> Text Document. Name the file MoviePlayer.txt and press ENTER. Note the two new panes that appear, the left one in DSL Input Mode, and the right one in Output Mode.

Enter the text "MoviePlayer" followed by a name (for example, "Superman") in the DSL Input Mode pane in “Intellipad”. Notice how the text is parsed and displayed as structured data in the dsl-preview - Output mode pane.

Add multiple lines of Movies and examine the parsed output, for example, the following.

MoviePlayer Superman
MoviePlayer "One Flew Over the Cuckoo’s Nest"
MoviePlayer WarGames

Experiment with the language by changing things like the MovieStart token. Note the generated error. Next, change the input file to reflect your changes.

Add New Patterns

In this task, you extend the basic language definition with additional information about movies, namely, the director of the movie. When adding code, you can compare what you did with the complete sample at the end of this procedure.

To the existing language, add a new token called By. Define the By token to be the constant string by, and make the token final.
Add a new syntax rule called Director. Define Director to be the By token followed by the Name token.
Add a second pattern with the or operator (|). In this pattern write the By token followed by the NameVerbatim syntax.
Create a third pattern with the or operator. This last one will ignore director using the empty keyword.
Update the Movie syntax by adding a new grammar term called Director referencing the Director syntax just after the MovieName term.
In the DSL Input Mode pane, update one of the MoviePlayer entries to include the by keyword followed by a value for a director.
```
MoviePlayer Superman by "Richard Donner"
MoviePlayer "One Flew Over the Cuckoo’s Nest" 
MoviePlayer WarGames by Badham
```
Notice that there are no parsing errors. However, the parsed output does not change. This is because we need to update the projection that produces the output.

The following code is a sample of the Mg language up to this point in the exercise.

module MoviePlayer
{
    language MovieLanguage 
    {
        interleave Skippable 
            = Whitespace;

        syntax Director 
            = By Name 
            | By NameVerbatim 
            | empty; 

        syntax Main 
            = movies:Movie+ 
            => Movies{ valuesof(movies) };

        syntax Movie 
            = MovieStart name:MovieName Director
            => name;

        syntax MovieName 
            = name:Name => name
            | name:NameVerbatim => name;

        syntax NameVerbatim 
            = '"' name:NameWithWhitespace '"' => name;

        token AlphaNumerical 
            = 'a'..'z' | 'A'..'Z' | '0'..'9';

        final token By 
            = "by";

        final token MovieStart 
            = "MoviePlayer";

        token Name 
            = AlphaNumerical+;

        token NameWithWhitespace 
            = (AlphaNumerical | Whitespace)+;

        token Whitespace
            = '\r' 
            | '\n' 
            | '\t' 
            | ' ';
    }
}

Add Projections

In the preceding task, the director content is not actually returned in the output defined by the projection. In this task, you further customize the language definition to add this to the output structure and to turn that output into a structure that can be consumed by the M compiler (and ultimately into T-SQL).

The complete code for this procedure is in the last step of the procedure.

In the Movies.mg file, change the Movie syntax by adding a variable director to the Director term.
Change the projection on the right-hand side of the Movie syntax. Define a set ({ and }). Within the set, add a label called Name followed by another set that contains the variable name. Also within the outer set, define a label called Director with another set that contains the variable director. In the Untitled input window, review the results of the current input data parsed with the new projection. The syntax for Movie should now look like this. Note that in the output pane, the Director label contains the keyword by, which is probably not what you want.
```
syntax Movie 
    = MovieStart name:MovieName director:Director 
    => { Name {name}, Director{director}} ;
```
Next, delete the by keyword in the output pane. Add a projection to the right-hand side of the Director syntax. Add a variable named name to the Name term and the NameVerbatim term in the first two patterns. For both of these patterns, the projection should return the variable name. For the pattern with empty, return the value "Unknown". Review the results of the current input data parsed with this new projection. The syntax for Director should now look like this.
```
syntax Director 
    = By name:Name => name
    | By name:NameVerbatim => name
    | empty => "Unknown";
```
Notice that the parsed output does not contain the by keyword.

The following code is a sample of the Mg language up to this point in the tutorial.

module MoviePlayer
{
    language MovieLanguage 
    {
        interleave Skippable 
            = Whitespace;

        syntax Director 
            = By name:Name => name
            | By name:NameVerbatim => name
            | empty => "Unknown";

        syntax Main 
            = movies:Movie+ 
            => Movies{ valuesof(movies) };

        syntax Movie 
            = MovieStart name:MovieName director:Director 
            => { Name {name}, Director{director}} ;

        syntax MovieName 
            = name:Name => name
            | name:NameVerbatim => name;

        nest syntax NameVerbatim 
            = '"' name:NameWithWhitespace '"' => name;

        token AlphaNumerical 
            = 'a'..'z' | 'A'..'Z' | '0'..'9';

        final token By 
            = "by";

        final token MovieStart 
            = "MoviePlayer";

        token Name 
            = AlphaNumerical+;

        token NameWithWhitespace 
            = (AlphaNumerical | Whitespace)+;

        token Whitespace 
            = '\r' 
            | '\n' 
            | '\t' 
            | ' ';
    }
}

Add New Data using the DSL

In this task, you compile the DSL, run an input file using the DSL loader to produce an M image, and then use that M image to load the SQL database with instances.

Save the Movies.mg file.
Run the M compiler with the MoviePlayer.mg grammar file that you have created. Set the package type to be Image.
```
"c:\program files\Microsoft Oslo SDK 1.0\bin\mg.exe" -p:Image Movies.mg
```
Run the Mg loader using Mgx.exe. Set the target to be M, the reference to be the image file created in the previous step, and the input file to be the sample MoviePlayer.movies.
```
"c:\program files\Microsoft Oslo SDK 1.0\bin\mgx.exe" /t:M /r:Movies.mgx MoviePlayer.movies
```
Note

To create an M model file that correctly matches the model defined in the first example, the file name must be named as MoviePlayer.xxx because in the current software the module name derives from the file name.
Run the output of the preceding operation into the M compiler. Be sure to include a reference to the image file created in the first exercise.
```
"c:\program files\Microsoft Oslo SDK 1.0\bin\m.exe" /r:Movies.mx /t:TSql10 /p:image MoviePlayer.m
```
Run the output of the preceding operation into the Mx loader. Be sure to include the /ig ignore command parameter to ignore loading the original Movie.mx content.
```
"c:\program files\Microsoft Oslo SDK 1.0\bin\mx.exe" /i:MoviePlayer.mx /ig /db:MovieDB
```

Query the Movies table in the SQL database using your favorite SQL query tool. You should see new rows in the Movies table for the new instances created using our MoviePlayer DSL:

Name
------------------------------------------------------------------------
Rocky II

Rocky I

Rocky III

Rocky IV

Rocky V

One Flew Over the Cuckoos Nest

Superman

 (7 rows affected)

You have accomplished the following: you have written a basic DSL for this domain and seen how that DSL can also generate data for the SQL database.

A DSL for Movies (Video)

Open a Basic DSL

Add New Patterns

Add Projections

Add New Data using the DSL

Additional resources