Jack Greenfield, Senior Director
Michael G. Lehman, Senior Architect

Microsoft Corporation

October 2008

Applies to:
   Software Factories

Summary: This article briefly recaps software-factory concepts, describes what has been happening with software factories, and introduces the next generation of software factories, which is called Blueprints. (9 printed pages)

Contents

Introduction
Software Product Lines
What Has Been Happening
What Is Next
Conclusion
Footnotes

Introduction

Have you ever wished that you could make your team more productive by codifying common conventions and automating frequently recurring tasks? Have you ever had to introduce new technologies or requirements? Have you ever wanted to package successful designs or implementations, so that you could use them again? Do you get tired of cutting and pasting between projects, and then fixing the same bug in all of the copies?

If so, you might be in the market for a factory. A factory is an accelerator for some type of deliverable, such as a Web service, rich Internet application, or service façade. It provides a process and resources that support its enactment, such as guidelines, patterns, templates, libraries, or tools. A factory is an SDK for a problem, instead of a product. Also, it is a domain-specific SDK.

Factories are really just another step in the quest to make software development more productive and predictable by codifying best practices, capturing design knowledge, raising the level of abstraction, and automating recurring tasks. Used effectively, they can reduce time to market, sharpen estimates, reduce defect levels, improve traceability, and assist with regulatory compliance.

Software Product Lines

Factories are a form of software product line (SPL). What is an SPL? You already know the concept, if not the name. Have you ever built multiple pieces of software—such as Web services, Web sites, or applications—that had a lot in common? No doubt, you have, and not by accident. No doubt, you try to achieve a measure of consistency from project to project, so that new projects take less time and existing projects are easier to maintain. No doubt, you think about your projects in terms of the key elements that they have in common, such as tables, stored procedures, pages, controls, and service endpoints. You think that way, because it helps you quickly to size up and identify risks in new projects.

What turns a set of projects into an SPL? It is the recognition that they can be treated as a group—not just as a random sequence of unrelated efforts—and that the resulting applications can be treated as variants of an archetype, with common features that are developed by using a common process that is supported by a set of common resources. If you have ever written a pattern, developed a framework, or built custom tools, the concept is intuitive. Developers have been harvesting, refining, packaging, and applying reusable assets since the dawn of the industry.

So, what is new about SPLs? They bring a disciplined approach to reuse—making it systematic, instead of ad hoc, by defining the type of deliverable that is targeted by the assets, describing the features and the architecture of the archetype, defining the development process, attaching the development process to the architecture, and then attaching the assets to the development process, to help the user find the assets when they are needed and understand where to apply them.

Software factories are an implementation of SPLs for the Microsoft platform, emphasizing model-driven development with Domain-Specific Languages (DSLs). You can think of them as “model-driven product lines.” Factories can target a wide variety of deliverable types, from systems to applications, services, layers, or components. They can be as simple as a document, or as complex as a development and runtime environment for oil-refinery management applications. They can span the software life cycle—from requirements to design, construction, testing, deployment, maintenance, or operation—or just cover a subset.

Figure 1 shows how factories are built and applied. As they work, developers identify opportunities to build new factories and improve existing ones. Then, they harvest assets from existing applications, refine the assets, package the assets into factories, and publish the factories for reuse. As they apply the factories, developers encounter defects and changes in requirements. Based on this experience, they identify opportunities to build new factories and improve existing ones, and the cycle repeats.

Figure 1. Illustration of how a software factory is built and applied

What Has Been Happening

A lot has happened with software factories. Here is a quick recap.

Patterns and Practices (p&p) developed an implementation technology, called the Guidance Automation Toolkit (GAT), and used it to build factories for common application types (for example, Web services, Web clients, smart clients, and mobile clients) and for common concerns (such as security and data access). The factories work together, thanks to common design elements, and they were shipped in source form, which let users customize them. These features led to wide adoption.

The key innovations in GAT and its runtime, called the Guidance Automation Extensions (GAX), were simple packages of Microsoft Visual Studio extensions and custom commands, called recipes, which prompt the user for input, and then perform a sequence of actions.

At around the same time, the Developer and Platform Evangelism (DPE) Platform Evangelism Team developed Project Glidepath for the micro-ISV community. Initially based on GAT/GAX, it created a new runtime to better address requirements that were important to the community (such as a small footprint, runtime extensibility, and RSS-based factory and runtime updates). The technology was used last year to build the S+S Blueprints, which is a collection of factories that are designed to help customers come up to speed quickly on Software-plus-Services (S+S). An updated version of the S+S Blueprints technology forms the foundation for Microsoft Blueprints, which is the next generation of factories.

In another development, the DSL Tools are shipping with Visual Studio Extensibility (VSX), where they support a new generation of customization mechanisms, and a new generation of UML modeling tools in Visual Studio Team Architect. The DSL Tools team is working also with a designer integration framework that implements a key piece of the software-factory vision by supporting model reference, query, navigation, trace, transformation, and refactoring.

The Software Factories Initiative, which is led now by the DPE Platform Architecture Team (PAT), is a common thread through these events. It is responsible for evolving the methodology and providing strategic direction to the many organizations—both inside and outside of Microsoft—that are working with software factories.

When we published the software-factories book, we said it was a 10-year vision. So far, we have seen several collections of factories, and several implementations of tool and runtime technology—with high user uptake and satisfaction, and strong demand for further progress. These efforts were just the opening, however, not the endgame. Like the Web services vision, the software-factories vision will be realized progressively over time by multiple generations of technology—each building on key learnings from its predecessors.

Figure 2 illustrates the evolution of factory technology to date:

Figure 2. Illustration of the evolution of factory technology to date

Here are some key learnings and requests to date from customers:

• Written guidance for the software-factories methodology.
• Factory support for team development in Visual Studio Team System and for SDLC phases beyond construction (such as requirements and testing).
• Easier factory authoring and maintenance, with fewer Visual Studio restarts.
• Many small ones that are easy to integrate instead of a few large ones, and a change in focus from developing individual factories to managing factory portfolios.

What Is Next

So, what is next for software factories? You already know part of the answer from the introduction. Blueprints are the foundation of the next generation of factory tool and runtime technology. What you might not know yet is that Blueprints are evolving rapidly, and that we are writing a handbook for the software-factories methodology that uses Blueprints technology.

Software-Factories Handbook

First, let us talk about the handbook. It will include the formal definition of the Blueprint deliverables; guidance for factory planning, authoring, and usage; a platform specification that describes the technologies that are required to apply the guidance; and a reference Blueprint with a pair of reference applications, cross-indexed to the guidance. The planning guidance will show developers how to implement SPL practices, such as domain analysis, feature modeling, and business-case analysis, and how to manage a factory repository.

So, what are Blueprints like? Let us have a look—first at the runtime, and then at authoring.

Blueprints Runtime

The Blueprints runtime is implemented by a small Visual Studio extension (VSX) package that is called the Blueprints Manager (BPM) and is available as a free download from CodePlex (http://www.codeplex.com/blueprints). During installation, it creates a repository. After installation, it downloads or imports Blueprints, and places them in the repository. Additional installations might or might not be required by a given Blueprint, depending on its contents.

The contents of a Blueprint fall into one of two categories: process and resources.

Process consists of one or more workflows, which are collections of related activities that step the user through actions on the deliverable that is targeted by the Blueprint. A workflow has programmatic elements and content:

• The programmatic elements include metadata that describes the workflow; an optional Windows Workflow Foundation (WF) implementation that defines the activities; optional preconditions that are evaluated to determine whether or not a given activity is ready to perform, given the current state of the Blueprint in a solution; and optional post-conditions that are evaluated to determine whether or not it can be marked as complete.

For example, the Blueprint author might specify that the user must choose delivery and storage platforms before choosing a user-interface technology; that is, choosing Microsoft Windows Mobile and Microsoft SQL CE would allow the user to choose Microsoft .NET Compact Framework, but not Windows Presentation Foundation (WPF).

• The content includes an overview, which provides an introduction to the workflow, and optional activity details. The content can be expressed as MHT files, which are authored by using Microsoft Office Word, and can be displayed by using any standards-compliant Web browser. Alternatively, it can be expressed as links to Web resources, such as blog entries or podcasts. It can also include “action links” that invoke resources within Visual Studio or inform the user that context-menu items are available within the solution.

For example, the details for one activity might include example code, while the details for another might point to a download on MSDN or CodePlex.

There are two types of resources: readables and executables.

• Readables are materials that users consume, such as Office Word documents, Office PowerPoint presentations, and videos. The documents might be freeform or structured (as, for example, guidelines, patterns, or checklists).
• Executables extend the design-time and/or runtime environment. Design-time extensions generally are Visual Studio extensions, such as snippets, templates, wizards, or tools (such as tools for DSLs). Runtime extensions might include libraries, frameworks, configuration files, databases, and other platform artifacts.

Some types of resources require installation and a Visual Studio restart, and some require only a Visual Studio restart; but most can be used immediately after being stored in the repository.

Each Blueprint is delivered in two forms: as a Blueprint Package that can be used inside of Visual Studio, and as a stand-alone Web site that can be viewed with any standards-compliant browser. Executables are available only inside of Visual Studio.

Blueprints can be downloaded via RSS feeds or imported from a Blueprint Package file that is stored on the local file system. RSS feeds might be provided by any authority that the user trusts. The BPM will remind you, as appropriate, when updated Blueprints might be available from the feeds to which you are subscribed when you open a solution that contains one or more Blueprints. The BPM can also be updated over RSS. The repository also is fully functional offline.

The BPM provides facilities for unfolding one or more Blueprints into a solution; it then presents the user with the process guidance that is provided by the Blueprints. It also exposes the resources that are provided by the Blueprints via links in the process-guidance and context menus. Users may use multiple Blueprints in the same solution in an ad hoc manner. In addition, authors may nest Blueprints—enabling a higher-level Blueprint to unfold automatically lower-level Blueprints to implement parts of its target deliverable.

The repository contains files that are organized by Entity (author) and Blueprint, including Visual Studio templates that are managed by the BPM [1], metadata that links commands to plug-ins, and two types of plug-ins: plug-ins that extend the BPM and plug-ins that implement recipes that are defined by Blueprints. A key feature of the repository is that it can be “frozen” in its entirety. The resulting snapshot can be associated with a solution. Later, when the solution is reopened, the snapshot is used to ensure version compatibility.

The BPM has a user interface. The Blueprints Explorer is a dialog box launched from the Tools menu. It offers a view of the repository, lets the user invoke resources, and provides access to the rest of the BPM user interface, including an Options dialog box, a Feedback dialog box [2], a Community dialog box (which provides access to blogs, forums, FAQs, podcasts, and the home page for the collection to which the Blueprint belongs), an About box, and a Workflow Window.

The dockable Workflow Window is the most important feature of the user interface. The process that is provided by a Blueprint contains instructions and information that guide the user through the activities that are necessary to build the deliverable.

Figure 3 shows the Workflow Window:

Figure 3. Illustration of Workflow Window

On the left is the list of activities that compose the workflow. The first item in the list is always an overview of the contents of the Blueprint. The list can be filtered to show the subsets that are ready, blocked, or completed, as Figure 4 illustrates. Blocked activities can be queried to show what must be done to unblock them. Activity details, if available, are shown on the right:

Figure 4. Illustration of Workflow Window, with activities and activity details

To see an example of the Blueprints Manager in action, visit the community landing page for Microsoft Blueprints at http://channel9.msdn.com/blueprints.

Blueprint Authoring

You can create your own Blueprints by using the freely available authoring environment that is called the Microsoft Blueprints Factory (MBF). The MBF is delivered as a Blueprint, which allows us to provide process guidance for Blueprint development, and resources that automate much of the process.

The process is simple, as Figure 5 illustrates. Create a Blueprints Factory solution in Visual Studio. Define the process for the type of deliverable that you want to target. Build resources to support the process. Add the process and the resources to the solution. Configure the Blueprint by using wizards that are provided by the MBF. Build the solution, generating the Blueprint Package and the stand-alone Web site. Deploy the Blueprint. Test it by building an example deliverable. Iterate as needed.

Figure 5. Illustration of Blueprint-authoring process

One of the most powerful features of the MBF is that a Blueprint can be developed in any solution—including its own test solution or a solution that contains a reference implementation from which you plan to harvest resources. As long as the Blueprint requires no installations or Visual Studio restarts, there is no need to close Visual Studio or open another instance.

Blueprints Road Map

How are Blueprints evolving? Blueprints 2.1 is being released at PDC 2008, and should be available by the time that this article is in print. Based on Visual Studio 2008 SP1, it builds on Blueprints 1.4, which was released to the community in September. Blueprints 2.1 contains many enhancements, including an integrated SQL CE repository, replacement of the anchor project by the Blueprints Explorer, an unlimited set of verbs, WF-based process guidance, and a WF-based recipe implementation.

Blueprints 3.0 is scheduled to be released with the next release of Visual Studio. The feature set is still evolving, but it is expected to include Visual Studio Team System integration, support for model integration, a WPF-based Wizard Framework, and support for new Visual Studio Extensibility mechanisms, including XCOPY extension installation and integration with the Visual Studio Gallery.

Plans beyond Blueprints 3.0 are not yet solidified, but currently they include targeting other Microsoft development and runtime platforms, such as Microsoft Office, Office SharePoint, and various online services.

Migrating to Blueprints

Given the large number of downloads of the p&p factories, the level of dependency that many users have on them, the fact that most users have customized them to some extent, and the large number of factories that have been developed outside p&p by using GAT/GAX, migration from GAT/GAX to Blueprints is an important issue. The Platform Architecture Team and p&p are working on it together, and will announce a migration strategy soon.

Blueprints Branding

Building on the success of the S+S Blueprints, the Platform Architecture Team has created a Blueprints master brand. The master brand supports sub-brands for collections of Blueprints. Sub-brands from Microsoft include Microsoft Developer Blueprints, Microsoft Architecture Blueprints, Microsoft Infrastructure Blueprints, and Microsoft Designer Blueprints. Additional sub-brands are in development.

The goal of the branding strategy is to enable developers to establish a distinct identity for a given collection of Blueprints, as a way to manage user expectations. Each collection has an owner, who defines its characteristics. Examples of characteristics include a given purpose (for example, to help customers come up to speed on S+S), a target user type or community (such as IT professionals, CodePlex users, and so on), a required runtime configuration (such as Microsoft BizTalk Server or Office SharePoint), and a standard table of contents.

Conclusion

After reading this overview of software factories and the new Blueprints implementation technology, we hope that you will download the Blueprints Manager and some of the Blueprints that are available both on CodePlex and over the default RSS feed. If you find them useful, we hope that you will consider developing some of your own by using the Blueprints Factory, and sharing them with the community on CodePlex. Also, please write to us and tell us what think. To learn more about Blueprints, please visit the Blueprints home page on the Platform Architecture Team site: http://msdn.microsoft.com/architecture/blueprints.

Footnotes

[1] Templates that are managed by Visual Studio reside in designated folders in the Visual Studio installation.

[2] The BPM also can collect and report anonymous feedback, if enabled by the user.