Auto-Generating Custom Solutions in Microsoft Visual Studio 2005

Jamie Laflen
Microsoft Corporation

May 2005

Applies to:
   Microsoft Visual Studio 2005
   Microsoft .NET Framework
   custom solutions
   Microsoft Windows Forms

Summary: Use new functionality available in Microsoft Visual Studio 2005 to develop custom code generation solutions for your organization. (13 printed pages)

Contents

Introduction
The Solution
Processing the XMI
Creating the Solution Structure
Code Generation
Client Code
Conclusion
Appendix

Introduction

In the fall of 2004 we started an engagement in which the customer wanted to develop a code generation solution based on the Microsoft Visual Studio 2005 Beta 1 Refresh. This solution was one part of a larger project in which we created architecture standards, guidance, and documentation to establish the .NET Framework as a first-class platform in this client's enterprise architecture. At first blush, we thought that the project would revolve around using Enterprise Templates and a C++ Wizard, as described in Enterprise Templates: Building an Application Construction Kit in Visual Studio .NET 2003. As we dug deeper into the requirements we realized that this approach would not work for this project, because the code generation portion required that users be able to import a model into a Visual Studio solution multiple times without losing work. An additional option was using Visual Studio's automation interfaces that are exposed to managed code through Primary Interop Assemblies (PIAs). After investigating the automation interfaces, we decided to develop the code generator as a multi-threaded Windows Forms application that uses the automation API and Visual Studio templates to generate a Whidbey-based solution. The generated solution also took advantage of new Whidbey features; in fact, it would have been much more difficult to generate the solution without generics and partial classes.

This document discusses the interesting points of the project and highlights the new Whidbey features that were leveraged to make this project a success. Early in the engagement, we convinced our client to commit to Visual Studio 2005 and Microsoft .NET 2.0 as they introduced .NET to the application domain of their comprehensive enterprise architecture. Only later, when we got further into the code generation solution, did we realize how much Visual Studio 2005 and .NET 2.0 would enable a more capable and robust solution.

The Solution

The Code Generator we needed to develop had to be able to generate the following application scenarios:

• Intranet smart client
  WinForms client connecting to the backend by means of Web services.
• Intranet Web application
  WebForms client connecting directly to backend services.
• Extranet Web application
  WebForms client connection to backend services by means of Web services.
• Web Services layer

One of the big benefits companies realize through using enterprise templates is the set up of the application structure for developers; this has the obvious benefit of making the code well-organized and consistent across applications. We modeled the architecture after the standard, recommended distributed application layered architecture.

The customer's modeling solution revolved around using Rational Rose to model the application business objects in UML and then export the model into XML Metadata Interchange (XMI) format. The customer already had a Java-based solution that consumed the XMI file and generated a sample J2EE Web application based on the model. The generated J2EE application allowed the developer to hit a sample Web page and perform Create, Read, Update, and Delete (CRUD) operations on the data types defined in the model (the sample model had Customer and Address data types). The customer's team we were working with liked this solution and wanted a similar solution for .NET technologies. At the beginning this seemed like a pretty daunting task.

As mentioned before, we needed to be able to generate four main application scenarios and generate classes in each application layer that reflected a business entity from the model. This broke down into the following tasks:

1. Process the XMI.
2. Generate the solution structure:
   1. Set up the directory structure.
   2. Create the projects.
   3. Set up the projects with appropriate settings, including the root namespace to reflect the hierarchical nature of the application.
   4. Add project references.
3. Generate CRUD classes in each layer that reflected the business entities provided by the model.
4. Generate client classes that can retrieve data through Web service calls or (in the case of the Web client) connect directly to the controller layer.
   1. If necessary, create the Web service proxy classes that will be used by the client(s) to connect to the backend services.

We also had the following overarching requirements:

• We needed to support both Visual Basic .NET and C# generated code solutions.
• The code needed to be able to import the same model into the same solution multiple times.
• Users could run the code generator without importing a model. In that case, the structure of the solution would be created without any code.

Processing the XMI

Processing the XMI input turned out to be extremely easy; we just delegated it to the Java application's command line interface, and had it return a schema.xsd file describing the business entities and a methods.xml file describing the behaviors contained in the model. We then used the classes in the System.Xml.Schema namespace to consume the .xsd file so that we "knew" what types were defined in the model; we used this information later when we created classes used to represent the types' CRUD operations. During this implementation we did not do anything with the methods.xml file other than process it, because (given the schedule) we could not figure out a good way to provide a default implementation for the methods while at the same time support the regeneration requirement without overwriting user code.

Creating the Solution Structure

The Visual Studio Solution's structure and directory structure needed to represent the namespace hierarchy. To build these structures we used project templates and the Visual Studio Automation API.

Visual Studio 2005 supports a new template format that is aimed at making it easy for developers to share code as templates. In previous versions of Visual Studio's templates, in addition to programming in Jscript and editing multiple configuration files, in order to use a template you had to elevate your user account's privileges in order to force a template to appear in the New Project dialog box. This made it difficult to deploy and/or share templates. You can see examples of the new template format within the following path:

[Visual Studio install directory]\Common7\IDE\ProjectTemplatesCache

As you can see by peaking into the .zip files, the template format has been greatly simplified to revolve around the .vstemplate file as the manifest for the template.

In Visual Studio 2005, users can download templates and those templates appear in the New Project dialog under "My Templates." The downloaded templates are installed by default within a directory in the user's ApplicationData directory. It is also expected that some corporations will modify this path to point to a network share allowing templates to be updated automatically (an offline mode is supported too). Also, later versions of Visual Studio 2005 are going to support a "make as template" option directly within the GUI. Our application did not use any of the default templates that came with Visual Studio, because they contained source files that had the wrong namespace set when the template was added to the Solution. This was because the code generator could only set the namespace on the project after the project was created.

**Note   **This is also a good practice because it makes it easier to add functionality to a template later without having to modify code.

We used the templates to represent all projects within all application layers; as might be expected, we most heavily customized the Web service and website templates because of the need to integrate with SiteMinder security.

Visual Studio 2005 replaces the old .etp projects that were used in previous versions with a new feature called SolutionFolders. SolutionFolders are a great addition, although we did wish that we could set the namespace on the folder as well as have Visual Studio create a physical directory to back up the folder and store the folder's contents.

To support these changes, the extensibility API includes a new interface aptly named Solution2 that exposes new methods to access and control these features. This includes a method to create SolutionFolders, as well as methods to look up project and item templates regardless of where they are installed on a user's machine. The methods to look up the location of the templates are needed because the full path to the template must be passed to the AddFromTemplate(...) method on the Solution2 or SolutionFolder interfaces.

In addition to the changes to the .etp files, Web Projects were also significantly improved in Visual Studio 2005. In this release, website projects do not have a project file, instead relying solely on information contained in the directory structure and in the solution file (.sln). This new implementation does have the following caveats:

• Website projects do not have an assigned namespace.

• All pages are JITed—the website does not need to be compiled.

• All shared code in a website project must be placed in the Code directory to be used by any pages.

• Pages can be defined in multiple languages (Visual Basic and C#) but only one type of language can be defined in the Code directory.

  **Note   **We had a problem with this in certain regen scenarios—in particular where the developer creates the solution structure on the first pass and then imports a model on the second pass. To work around this issue, we placed a bootstrap.cs/bootstrap.vb file in the Code directory dependant on which website template the developer chose on the first pass.

This new functionality can be programmatically manipulated through the VsWebSite.Interop.dll assembly. This extremely handy assembly allows you to:

• Start the webdev.webserver.
• Discover the base URL to the server.
• Enumerate all Web services in the project.

After this phase was completed we had generated a Visual Studio Solution with SolutionFolders, a placeholder representing portions of the namespace without a project. The Visual Studio templates were a combination of Web, Class Library, and Executable project templates. If the solution were opened it looked like the following:

Figure 1. Solution

On the file system, the generated directory structure looked like the following:

Figure 2. Generated directory structure

Code Generation

Once the structure of the solution was created we needed to populate the projects with code that represented the business entities imported from the model. An example will help make this clear—say a model specified a Customer business entity in the .xsd file returned by the J2EE XMI processor. The generator would use xsd.exe to convert the .xsd file into a code file containing the Customer business entity and add this file to the Entities project. It would then add classes to the Web service façade, controllers, and data access projects as follows:

**Note   **The InMemDa is an in-memory implementation of the DataAccess layer so that the CRUD operations can be performed without Database tables. The selection of which data access class to use is controlled by conditional compilation statements in the user code (not .g.) and defaults to in-memory for Debug with the database-backed data access class selected for a Release build.

As you can see, the code generator added two (or in the case of the Web service façade project, three) files to a project to reflect one business entity. The code generator was implemented this way to support the reimportation of a model into a previously generated solution. The new partial classes feature in Whidbey made this possible by enabling us to split a class's definition across multiple files; that is, CustomerDa.cs and Customer.g.cs. In our implementation, the '.g.' extension indicates that the file is a generated file and should not be modified by user code because the modifications will be overwritten the next time the model is imported. When adding these files to their respective projects we made the generated code dependent on the user code so that the generated code was hidden by default but was displayed when the user pressed the "Show all files" button:

Figure 3. Generated Class

As stated above, the generated code needed to support the basic CRUD operations for each business entity. In previous versions of Visual Studio, we would have had to duplicate a lot of common code to implement this requirement; however, in Whidbey, we were able to use generics to implement type-safe CRUD operations once for all classes in each layer. Each generic definition took two parameters: the first, the name of the business entity type; the second, the name of the primary key type. For example, the DataAccessBase class definition was the following:

public class DataAccessBase <T, K> : 
  ICRUDOperations<T, K> where T : new() {...}

As you can see, the DataAccessBase class implements the ICRUDOperations interface; this interface is implemented at all layers (WSFacade, Controllers, DataAccess) to cement the understanding that all layers implement CRUD operations. The interface definition is as shown:

public interface ICRUDOperations<T,K> where T : new() {
   void Create(T toCreate);
   T Read(K id);
   T[] ReadAll();
   void Update(T toUpdate);
   void BatchChange(Batch<T> updates);
   void Delete(T toDelete);
}

The implementation of the CustomerDa class follows directly from the previous definition:

CustomerDa.cs

public partial class CustomerDa {...}

CustomerDa.g.cs

public partial class CustomerDa : 
      CodeGen.Server.DataAccessBase<Customer, System.String> {...}

In addition to the basic CRUD operations, the generic in the WSFacade layer also supplies default authorization and error handling/exception mapping. Developers can plug in their own authorization functionality by overriding GetAuthZ() and supplying their own implementation of the following interface:

public interface ICRUDAuthZ <T,K> {
   bool IsAuthCreate(T toCreate);
   bool IsAuthRead(K pk);
   bool IsAuthReadAll();
   bool IsAuthUpdate(T toUpdate);
   bool IsAuthBatch(Batch<T> changes);
   bool IsAuthDelete(T toDelete);

   void LogAccessDenied();
}

**Note   **The developer can also subclass CRUDAuthZBase<T,K> and override any specific methods.

They can also override the ProcessInternalException(...) method for mapping exceptions. Validation is performed at the Controller layer, but developers can plug in their own implementation by overriding GetValidate() and supplying their own implementation of the following interface:

public interface ICRUDValidate< T, K> {
   bool IsValidCreate(T toCreate, out string msg);
   bool IsValidRead(K pk, out string msg);
   bool IsValidUpdate(T toUpdate, out string msg);
   bool IsValidBatch(Batch<T> changes, out string msg);
   bool IsValidDelete(T toDelete, out string msg);
}

**Note   **The developer can also subclass CRUDValidateBase<T,K> and override any specific methods.

In this implementation, the CodeGen projects that supply the generic classes (as well as other supporting classes) are added as projects rather than as assemblies to allow developers to see how the code was implemented, as well as to make it easier for developers to step through the code. On additional imports, the CodeGen projects are overwritten, providing the ability to release bug fixes to the CodeGen projects.

Client Code

The backend code-generation was relatively straightforward. The client code was a little more complex because it involved generating Web service proxies, dynamically binding to CRUD operations as well as dynamically binding to business entities. Since we "knew" what the CRUD methods were on the Proxy or Controller classes we were able to dynamically bind to a Proxy or Controller instance at runtime.

In the case of the WinClient, we actually generated code to fill in the left tree containing all the business entities and provide the correct Web service proxy. In the Web Client case, we used the new Master Page feature in ASP.NET and created one CRUD page per business entity. Both approaches have their plusses and minuses. In the WinClient case, if a developer regenerates a solution he or she must reimport all models. The developer does not have to do this with the WebClient, but the WebClient has many more files.

When generating the Web service proxies we wanted to use the new sharetypes wsdl.exe switch to avoid the same type being defined multiple times (for instance, two different .asmx pages have methods that return the same type). Since this feature is not available through the Visual Studio "Add Web Reference" API we had to make a call out to wsdl.exe in order to generate the Web service proxies. The other problem was how to generate the WSDL for all the generated Web services. A great new feature in Visual Studio 2005 is the webdev.webserver—a small Web server that Visual Studio uses to host ASP.NET projects. The API for this Web server is exposed as a managed .dll allowing us to start the server, enumerate all the Web services on the website, and make the wsdl.exe call specifying all the URLs on the command line.

As we did in the previous section, we factored most of the code out into separate CodeGen projects using generics where possible.

Conclusion

This code generation solution was a major component of our deliverables to this client. The client's internal architecture guidance organization had a strong commitment to model driven architecture (MDA), especially the use of UML and code generation templates and tools to accelerate the start of projects and to encourage the use of common patterns and frameworks. New features of Visual Studio 2005 and .NET 2.0 gave us the ability to more quickly build a code generation solution that integrated well with Visual Studio, and by taking advantage of .NET features such as generics and partial classes completely fulfilled, and in some facets exceeded, the client's requirements.

Appendix

Calls to External Executables

We ended up shelling out three times during the code generation process. Because of how the external executables interacted with .NET we found the following extremely helpful when making these calls:

• Setting timeouts on the calls—there were cases in which the call to run the executable did not return.
• Using the new ErrorDataReceived and OutputDataReceived events on the process class to handle data from the process as it was output.
• Checking that the expected output actually existed—sounds simple, but it made a great sanity check when depending on external code that may or may not have a non-zero return code.

Technologies Used

This section delves more deeply into the technologies used by the code generator as well as the generated solutions. Links to MSDN are provided where possible. The technologies used can be broken into two parts—those used by the code generator itself, and those used by the generated solution.

Code Generator

Extensibility API

Automation and Extensibility for Visual Studio

Visual Studio has a complete managed API that allowed us to automate the creating and reading of solutions, projects, and project items.

Solution Folders

Using Solution Folders

New to Visual Studio 2005 are SolutionFolders. SolutionFolders take the place of the .etp project files in previous Visual Studio editions.

Visual Studio Web Server

Web Servers in Visual Studio

A stand-alone Web server is included with Visual Studio 2005. This Web server eliminates many of the security problems previously associated with doing ASP.NET development—debugging and attack surface. A managed API exposes the ability to control the webdev.webserver as well as enumerate the Web services contained in a Web project.

Visual Studio templates

Templates for Projects and Project Items

Templates have been significantly revamped and enhanced in Visual Studio 2005. One goal was to make it easy for developers to share templates with each other. Towards this end, the new templates can be packaged as zip files and stored in an end-user controlled directory. Previously, templates were typically installed inside the Visual Studio install directory or inside program files (with a HKLM key pointing to the installation point).

IWizard interface

IWizard

Although not used in the project, it was investigated as a replacement to the sub-wizard functionality in previous versions of Visual Studio templates.

Anonymous methods delegates

Anonymous Methods

These new delegates were used to stream-line code.

Web Services Description Language Tool (Wsdl.exe)

Wsdl.exe

wsdl.exe was used to generate Web service proxies that share types. URLs for all CRUD Web services were placed on the command-line when making the call to the executable. A .bat file with the command was placed in the solution so that the proxies could also be regenerated by the developer.

XML Schema Definition Tool (Xsd.exe)

Xsd.exe

xsd.exe was used to generate source code reflecting business entities described in a .XSD file. The .XSD file was returned from the XMI processor after successfully processing the model.

Generated Solution

The following framework features are leveraged to create the generated solution:

Generics

Generic base classes

The solution made heavy use of generic base classes to provide CRUD functionality in a type-safe manor. Each base-class had two parameters: the first, the business entity type; the second, the type of the primary key used to look up an instance. Generics were also used to perform type-safe binding to CRUD methods.

Partial Classes

Partial type definitions

Because of the requirement to be able to regenerate a solution, partial classes were used to physically separate user and generated code. This also made it potentially easier to upgrade solutions to newer versions of the template.

Reflection

Reflection Overview

Reflection was used by the Data Access layer to detect public GET and SET properties that were used in the generated parameterized SQL. Reflection was also used by the Web and Win clients to dynamically bind to CRUD methods.

Windows Forms

DataGridView

DataGridView Control (Windows Forms)

Allowed dynamic binding to different business entities.

ASP.NET Web Forms

The ASP.NET Web Forms client used the following new controls/functionality:

Masterpages

Understanding ASP.NET Master Pages

One master page is supplied with the Web Client template. It is used to provide a consistent look and feel across all CRUD pages.

Themes

Understanding ASP.NET Themes

A theme is provided for the template.

Sitemap

Understanding ASP.NET Site Navigation

The new Sitemap functionality is used to consume a generated .xml file to provide dynamic site tree/path features.

GridView

Introduction to the GridView Control

Used to bind to the provided business entity for Read, Update, and Delete features.

DetailsView

Overview of the DetailsView Control

Used to bind to the provided business entity for Create features.

ObjectDataSource

Overview of the ObjectDataSource Control

Used to encapsulate access to the CRUD operations.

Building blocks

Although Enterprise Library was not yet available for .NET framework 2.0 we did compile it against that version of the framework.

patterns & practices: Enterprise Library: Workspace Home

The following blocks were used

• Enterprise Library: Data Access
• Enterprise Library: Logging

© Microsoft Corporation. All rights reserved.