December 2011

Volume 26 Number 12

ALM Rangers - Visual Studio ALM Rangers -- Rise of the VM Factory

By Willy-Peter Schaub, Brian Blackman | December 2011

In this article, we delve into the Virtual Machine Factory (VM Factory), exploring its concepts and advantages.

This article is part of a series in which the Visual Studio ALM Rangers present guidance to assist you in solving problems in complex, real-world environments. Our goal is to help you improve the consistency and quality of your solutions and your overall Application Lifecycle Management (ALM) process.

To recap, the Rangers are a group of experts who promote collaboration among the Visual Studio product group, Microsoft Services and the Microsoft Most Valuable Professional (MVP) community by addressing missing functionality, removing adoption blockers and publishing best practices and guidance based on real-world experience.

The VM Factory (vsarvmfactory.codeplex.com/) is the reference implementation of a software solution and associated guidance that automates the creation of virtual environments using a nearly fully automated and consistent factory strategy. The purpose of the VM Factory is to develop prescriptive guidance around virtualization, particularly for Visual Studio, Team Foundation Server (TFS), Lab Management and associated prerequisites. The main objectives, as shown (with corresponding numerals) in Figure 1, are: (1) flexible schema-driven automation; (2) a simple one-click virtual machine (VM) creation scenario; (3) a reusable factory that automates the installation and configuration of physical and virtual environments; (4) consistent and repeatable environments; (5) the ability to reuse preconfigured environments.

Figure 1 VM Factory Objectives

What the VM Factory objectives do not pursue is a reduction in build and configuration time. This is called an anti-pattern.

Although the VM Factory is far more efficient and disciplined than a human counterpart, the bulk of the environment generation time is made up of actual installation time, which can’t be noticeably reduced by automation. The VM Factory initiative emerged from the Rangers ecosystem, which is renowned for the distributed, disconnected and often poor bandwidth environment constraints that it operates within (see the September 2011 article, “Visual Studio ALM Rangers—Reflections on Virtual Teams,” at msdn.microsoft.com/magazine/hh394152). Even though sharing and downloading VMs over the Internet is feasible and is an accepted practice, it can be a challenge in locations where bandwidth is an expensive and scarce resource. Instead of investing days, or possibly weeks, to download a VM, the VM Factory allows identical images to be recreated remotely using local resources. Despite the limitations that local resources might have, these recreated images are delivered with minimal or no user intervention

The delivery of consistent images is another core advantage we often forget to mention, but it’s one of the main reasons the Rangers use the VM Factory. The image schemas flow from version to version, which means that when we create a new version (v1.1) to fix an issue, then another build to include a new software version (v2), we know the fix and the new software will flow to subsequent versions. This is especially helpful when the next version (v3) might not be built for months or even years. When that much time elapses between versions, a different team might be assigned to the project. If consistent images are in place, the VM Factory Task Sequence configuration and structure setup acts as a living document with which the new team can work. 

Last but not least, the VM Factory allows on-demand image creation. This reduces the disk storage requirements on both desktop and portable computers, as well as the “angst” about whether an image pulled from storage is the most up-to-date image.

Virtualization within Lab Management provides an automated workflow for build, deployment and testing of software. Creating these virtual environments requires a lot of manual effort. You currently have to set up a library of VMs and templates for Microsoft System Center Virtual Machine Manager (VMM). Additionally, you have to work within a Lab Management topology.

Using the VM Factory significantly reduces the effort required to create VMs and VM templates. More importantly, the VM Factory makes it easier to change an OS or an application that’s used for a VM or VM template. The VM Factory is easy because you only have to change a task sequence and then execute the automated recreation of the image. In the VM Factory, these are known as golden images and function as source VMs for Lab Management.

When you create the VM, you’ll be prompted to select the task sequence to execute. We provide two task sequences for Visual Studio ALM Rangers Lab Management, both of which we’ll cover in more detail later.

Factory Architecture at a Glance

As shown (with corresponding numerals) in Figure 2, the VM Factory architecture can be hosted on (1) any Windows platform with minimal prerequisites, such as a minimum 1GB of RAM and 100GB of free disk space dedicated to the VM Factory. The architecture is based on the (2) Microsoft Deployment Toolkit (MDT), one of the Microsoft solution accelerators (bit.ly/tp5QX4). The toolkit provides a (3) common administration console and comprehensive tools and guidance to define and manage deployment of OSes and applications. Using (4) task sequences, we define the sequence of installation and configuration steps that are required to deploy an OS and applications, such as Microsoft Office and Visual Studio, (5) use of custom scripts such as the Windows PowerShell scripts that are covered later in this article, and (6) referencing the media for the OS and applications to be used. Currently, the VM Factory assumes that all the media is physically present in the VM Factory. However, we’re researching options that will allow us to reference other media sources, such as MSDN downloads, to deliver greater flexibility and reuse of existing resources.

Figure 2 VM Factory Architecture

Each VM Factory comes with (7) Litetouch CD Boot ISO images. These allow you to initiate a new installation and connect to the deployment share hosted on the VM Factory. During the boot and installation process, the (8) appropriate task sequence is selected. The task sequence defines the software, configuration and the scripts that are applied to the (9) target machine during the Litetouch guided installation.

The Visual Studio ALM Rangers VM Factory is a prepopulated MDT instance with predefined Factory Task Sequences. These task sequences build on top of the MDT task sequence templates for new machine deployment. They define all steps necessary to create a new machine from bare metal (or bare virtual metal in case of the VM Factory). Starting with formatting the drive and installing the OS, the steps continue up to (and including) the installation of applications.

The task sequences included in the VM Factory are used by the Rangers to build images for demonstrations, proof of concepts, hands-on labs and testing purposes. Following are descriptions of these task sequences:

• Team Foundation Server base images task sequences define all-inclusive Visual Studio images containing both TFS client and server features. Also included are Word 2007, Excel 2007, PowerPoint 2007 and, optionally, Microsoft Office SharePoint Server (MOSS) 2007. Task sequences for both x86 (based on Windows Server 2008) and x64 (Windows Server 2008 R2) are included in the VM Factory.
• Lab Management Hands-on Lab image task sequence defines an all-up demonstration environment for Lab Management. It’s based on the x64 TFS base image, but also contains Active Directory, DHCP and VMM. DHCP is fully configured but disabled for security reasons. VMM and Visual Studio Lab Management are both completely configured and will only require that you add a domain-joined Hyper-V host to the default host group in VMM.
• Lab Management base image task sequence is useful to create images to test your software using Visual Studio Lab Management. It defines an image based on Windows Server 2008 R2 and installs Visual Studio Test Agent, Visual Studio Test Controller, Visual Studio Lab Management Agent and Visual Studio Team Foundation Build. Most test configurations will require a combination of these agents.

When you set up your own factory, you can modify these sequences to your needs or use them as a base for your own custom task sequence. For more information about how to do this, see the “Administration” section later in this article.

Automation with Windows PowerShell

The VM Factory contains Windows PowerShell scripts developed as part of the Ranger factory initiative to automate the setup, configuration and tuning of the Visual Studio ALM Rangers base images. In addition to the Windows PowerShell scripts and CMD files, we make use of other Microsoft services or product applications such as ntrights.exe, stsadm.exe, psconfig.exe and iisreset.exe.

Initial factory plans called for only Windows PowerShell scripts. However, we adopted MDT as the Visual Studio ALM Rangers base image factory and call Windows PowerShell scripts to perform special configuration, tuning and tweaking of the MDT-based setup. We’ll cover some important notes and a few snapshots of the scripts, directing you to the VM Factory for the full source.

To run the Windows PowerShell scripts successfully, you must have Windows PowerShell installed. The scripts are designed to run on Windows Server 2008 or later versions. Because the scripts aren’t signed, and elevated privileges are required to make system changes, you must run the Windows PowerShell command “Set-ExecutionPolicy Unrestricted.”

If you use PowerGUI from powergui.org, which is a GUI and script editor for Windows PowerShell, the scripts won’t run in debug mode. You must run them in Windows PowerShell proper or by using Alt+F5 while using PowerGUI. And finally, the 1-pre.ps1 script requires NTRights.exe, which is installed from the Windows Server 2003 Resource Kit. Without NTRights.exe, you’ll have to disable the calls at the last few lines of the script in the function createTFSAccounts and manually set Interactive Logon rights for the TFSSERVICE and TFSBUILD services accounts (see Figure 3). You can download the Windows Server 2003 Resource Kit Tools from: bit.ly/fbafh.

Figure 3 Use of NTRights.exe in a Windows PowerShell Script

The pre- and post-factory installed Windows PowerShell scripts contain several functions that use various objects and type adapters, such as the Active Directory Services Interface (ADSI) type adapter shown in Figure 4.

Figure 4 Use of the Windows PowerShell ADSI Type Adapter

The Windows PowerShell ADSI type adapter is used to create user accounts, set passwords and set the admin password to never expire. There’s liberal use of the Set-ItemProperty and New-ItemProperty Windows PowerShell cmdlets to manipulate the registry for the VM Factory, such as enabling Prefetch, setting program priority and disabling some of the Windows Server 2003 reliability features. Making these changes to the registry minimizes manual configuration tasks that are required to achieve the benefits of automation, as shown in Figure 5.

Figure 5 Manipulating Shutdown Event Tracker

As we mentioned earlier, we recommend that you explore the full source code so that you can understand how we use Windows PowerShell to automate many other tasks such as disabling the screen saver, disabling the inbound firewall, enabling remote desktop, disabling IE Enhanced Security, enabling super fetch and changing the server name.

Administration

Administration of the VM Factory is straightforward. Because the VM Factory is basically a prepopulated MDT deployment share, all possibilities of the MDT are at your disposal for customization or extension. The main tool for administering MDT deployment shares is the Deployment Workbench. The Help topics document the Deployment Workbench very well, but print-ready documentation can be downloaded from www.microsoft.com/en-us/download/details.aspx?id=25175. We’ll provide a short description to get you started.

When you start the Deployment Workbench, you’re presented with the familiar Management Console UI as shown in Figure 6. In the left pane, expand the Deployment Shares node. If you followed the instructions from the guidance on CodePlex correctly, the “VSTS Rangers Deployment Share” should be available. If not, you can open it now by using the appropriate action on the rightmost pane.

Figure 6 Deployment Workbench Administration Interface

Let’s look at Figure 6. By expanding the VSTS Rangers Deployment Share, you can browse through the contents of the deployment share. This is a user-friendly view of the XML files in the Control directory of the deployment share directory on disk. Applications, Operating Systems and Task Sequences are our main points of interest. Whenever you make changes to the Deployment Share settings, you should select Update Deployment Share. This will regenerate the boot images that you use to start any of the task sequences.

Application definitions describe where the source files for an application are located in the Deployment Share and how to install the application using the Quiet install command, as shown in Figure 7. The command shouldn’t require any user interaction to complete the installation. Sometimes this requires that you prepare the setup by creating an unattended installation script. Where necessary, the Rangers have already done that for the applications in the VM Factory.

Figure 7 Task Sequence Task Details

For OS definitions, the MDT is able to extract all necessary information from the OS installation source. Usually, one set of source files will contain files for multiple versions of an OS. All the different versions will be listed under Operating Systems and can be selected for installation in Task Sequences.

The task sequences included in the VM Factory, described earlier, are located under the Task Sequences node. Viewing the properties of a task sequence lets you modify the task sequence itself. Figure 8 shows that task sequences are divided into several phases. The important phases are the Install phase, which installs the OS, and the State Restore phase, which handles all post-OS installations. The toolbar provides options to add or remove steps from the task sequence. MDT provides several different steps. Figure 8 also shows the properties of an Install Applications task.

Figure 8 Task Sequence Properties

Extending the VM Factory

Extending the VM Factory is also straightforward. Let’s look at the basic steps to add another tool to the image. As an example, we’ll add the Code Metrics Power Tool (microsoft.com/en-us/download/details.aspx?id=9422) to the task sequence. The tool is installed by an executable called MetricsSetup.exe. The process consists of the following steps:

1. Determine quiet installation command line for the application
2. Copy application to Deployment Share
3. Start MDT Deployment Workbench
4. Create new Application Definition
5. Add application to Task Sequence
6. Test updated Task Sequence

Step 1 is to find out how to install this tool without user interaction—an unattended installation. We could look at the product documentation for unattended installation information. Another method that usually gives us a hint is executing the file with /? or -? as an argument. When we run MetricsSetup.exe /? from the command line, the Usage dialog box in Figure 9 is shown. We verify that the quiet install command line is MetricsSetup.exe /q by running this on a test computer.

Figure 9 Result of Running MetricsSetup.exe /?

Step 2 is to move or copy the software to the correct location in the deployment share. To be consistent with the rest of the software in the deployment share, we create a directory called “C:\Deploymentshare\INSTALL – Code Metrics Power Tool\Source\” and copy the executable to the new directory.

Now that the files are in the correct location, we start the MDT Deployment Workbench (step 3) to add the application definition (step 4). From the MDT’s initial view (see Figure 6), we find and select the Applications node in the left pane and choose New Application. A wizard helps us create the new application definition. On the first page of the wizard, we choose “Application without source files or elsewhere on the network,” because we already moved the executable to the correct location.

On the next wizard page, we need to enter the Application Name that will be shown in all the interfaces and some additional, optional information that’s not used in the process. The last wizard page in the application definition process is to enter the quiet install command and the source directory (see Figure 10). After finishing the wizard, you’ll find the new application in the application list.

Figure 10 New Application Wizard: Defining Command Details

To actually install the application, we need to add a step to a task sequence (step 5). In the task sequence editor, find the location where you want to add the install action. On the Add menu on the toolbar, select General | Install Application to add a new action to the task sequence. In the properties pane on the right, select “Install a single application” and click Browse to select the Code Metrics Power Tool application. Click OK to save the task sequence. We’re ready for testing (step 6).

The Latest Guidance

Now you know everything you need to know to start tweaking your deployment share to your specific needs.

You can find the latest guidance and deployment share templates on CodePlex (https://www.codeplex.com/) and a table of contents of associated blogs, videos and other information on the Rangers blog (bit.ly/qEIRPl).

Having proven the concepts and the technical feasibility of the automation and consistency offered by the VM Factory, we’re planning the following improvements:

• An easier implementation and administration of new factories
• Friendlier and controlled maintenance of existing factories, following the great principles of the Deployment Hydration initiative
• Ability to schedule factory automation
• Ability to use existing media libraries, such as MSDN downloads, instead of duplicating the media in each factory

Stay tuned as we continue to enhance this vital tool.

Brian Blackman is a principal consultant with the Microsoft Services Partner ISV team, focusing on affecting ISV partners’ success in engineering and in the marketplace. He is a CSM, MCSD (C++), MCTS and Visual Studio ALM Core Ranger. He spends his time writing code, creating and delivering workshops and consulting in various concentrations and all things ALM.

Paul Meyer is a senior application platform consultant with Microsoft Services in the Netherlands, with a strong focus on development processes and tooling. He has been involved in Visual Studio ALM Rangers projects for several years.

Willy-Peter Schaub is a senior program manager with the Visual Studio ALM Rangers at the Microsoft Canada Development Center. Since the mid-’80s, he’s been striving for simplicity and maintainability in software engineering. His blog is at blogs.msdn.com/b/willy-peter_schaub and he’s on Twitter at twitter.com/wpschaub.

Thanks to the following technical experts for reviewing this article: Vladimir Gusarov, Bill Heys, Bijan Javidi, Zayd Kara, Vijay Machiraju, Robert MacLean, Rui Melo and Patricia Wagner

About the Author