Agile Development

Extend Team Foundation Server To Enable Continuous Integration

Ben Waldron

Code download available at:TeamSystem.exe(173 KB)

Contents

Agile Methodologies
Continuous Integration
Visual Studio 2005 Team System
Project Unit Testing
Creating a Team Project
Creating the Build Unit Tests
Defining the Build
Extending Team Foundation Server
Assigning a Work Item
Conclusion

Many development teams have adopted "agile" methodologies to manage change and to improve software quality. These methodologies promote continuous integration as a practice to build and test software products incrementally as new features are included, bugs are fixed, and code is refactored. So how does Visual Studio® 2005 Team System and Team Foundation Server facilitate the process of agile development and continuous integration?

This article answers that question by creating an example project using agile concepts such as test-driven development (TDD) using the new unit testing features in Visual Studio 2005 Team System. After the project is completed, I'll show how to create a team project using Team Foundation Server and use this technology's extensibility features to build a custom Web service that enables continuous integration to build the application as code is checked into source control.

Agile Methodologies

Before drilling into specific agile development concepts, it is useful to revisit the history of agile development and define what makes a specific methodology agile.

Several methodologies were born in the 1990s as alternative approaches to process intensive development models like the Rational Unified Process (RUP). Some developers felt that these models did not represent the dynamic nature of software development where priorities often change and user feedback is critical. Through experience, many development teams recognized that software projects are built in many shapes and sizes, and that heavyweight software methodologies do not fit in many cases, especially for smaller, more dynamic development efforts. The predictive waterfall methodology, for example, does not scale well for rapidly changing development environments.

The extreme programming, adaptive software development, and Dynamic Systems Development Method (DSDM) methodologies were all created to allow project management and developers to adapt to change quickly without sacrificing quality. In retrospect, it was clear that these methodologies had many common themes, such as the ability to respond quickly to customer feedback and business demands, an understanding that customers have a strong voice in how the product evolves, short development iterations, and full testing coverage. Development teams began adopting the methodology that was the best fit for their organization and fed improvements back to evolve the methods over time.

Resources for Agile Development

Agile Development
Manifesto for Agile Software Development
Agile Alliance

Extreme Programming
Extreme Programming: A gentle introduction
Extreme Programming Wiki

Adaptive Software Development
Messy, Exciting, and Anxiety-Ridden Adaptive Software Development

Dynamic Systems Development Method
DSDM Consortium

Scrum
Jeff Sutherland's Blog
Control Chaos

Microsoft Solutions Framework
MSF for Agile Software Development

In 2001, a group of wise software professionals convened to discuss commonality among these methodologies—a meeting of the "five families" of adaptive software development. These software professionals coined the term "agile" to encompass all that was in common among the methodologies.

Since that time, agile development has grown and tools have been created to help. Meanwhile, development teams are reaping the benefits. Unit testing tools like NUnit have become integral parts of the development process. Likewise, continuous integration has become a standard practice that many development teams have adopted even without fully adopting an agile methodology. For more information on this topic, take a look at the "Resources for Agile Development" sidebar.

Continuous Integration

Continuous integration is a simple, powerful technique that Martin Fowler promoted as a best practice in software development, and a technique that fits easily into an agile development process (see Continuous Integration). Continuous integration takes the notion of a daily build a step further by building a software product each time any unit of the software changes. In most cases, a build is created each time a developer checks any code or configuration change into a source control. The build takes all files from the central source control repository and compiles the build. The status of the build is communicated to developers and testers on an ongoing basis so that failed builds can be caught quickly and corrected. Variations on this take it one step further, and perform the build before the check-in is finalized, allowing the check-in to be aborted if it would cause a break.

Extreme programming in particular emphasizes TDD, a natural fit into continuous integration. TDD proposes that the developer write a test case (typically a unit test), write the code exercised by that test, run the automated test, and refactor the code as necessary. Writing these tests effectively yields unit tests that give adequate test coverage across the product. These automated tests are placed in a common area where all developers and testers have access to run them before checking any code into the build.

This fits in well with continuous integration because, after a build is compiled, all unit tests are run on the latest build. If any unit test fails, then the build fails because something is not working properly. It is very important that the test cases cover as much of the product as possible to ensure a quality product. Another good practice is to produce unit tests for bugs that are fixed so that bugs can be verified and checked on an ongoing basis in an automated way.

One obvious benefit of continuous integration is transparency. Failed builds and tests are found quickly rather than waiting for the next build. The developer who checked in the offending code is probably still nearby and can quickly fix or roll back the change. If test cases are plentiful and thoughtfully composed, there are fewer burdens on quality assurance (QA) resources to find latent bugs, which gives you peace of mind regarding software quality. The result is more time for developing features and refactoring and less time spent trying to find integration errors.

Although the remainder of this article focuses on unit testing and continuous integration using Visual Studio 2005 Team System and Team Foundation Server, it is certainly possible to implement continuous integration without these tools. CruiseControl.NET is another example of a tool you can employ to implement continuous integration for your projects. However, I will show you how Team Foundation Server can automate builds to integrate seamlessly with the new unit-testing features in Visual Studio 2005.

Visual Studio 2005 Team System

Much has been written about Visual Studio 2005 Team System and its new capabilities for development teams. A good resource for learning about these features is the article, "Get All Your Devs in a Row with Visual Studio 2005 Team System" by Chris Menegay.

Team Foundation Server is the back-end server component that drives much of the Visual Studio Team System functionality. The feature that jumps out is the new source control system. This can replace Visual SourceSafe® and it even has a nice conversion utility to migrate your existing source code. A new version of Visual SourceSafe has also been released if you choose not to migrate.

Team Foundation Server has much more to offer than source control. You can track work items such as bugs, tasks, and project documentation. These items are stored and versioned, and are considered as valuable and as necessary as source code. Items that all team members, not just developers, need on a daily basis to complete tasks are stored within Team Foundation Server and presented through the Visual Studio user interface, allowing you to have this information in the tool you use most.

Team Foundation Server is driven by Web services that handle the interaction with Visual Studio clients. SQL Server™ 2005 is used to store work items and maintains sources as compressed reverse deltas from version to version. SQL Server Analysis Services and Reporting Services provide project-related reports for test results, metrics, and code churn. Windows® SharePoint® Services provides a portal for project collaboration and document repositories. The project team site lets team members who do not work in Visual Studio have access to project information and resources.

Source code is compiled centrally using the Team Foundation Build Server which can be installed on a standalone server. Code is checked out from the source control system, built, and tested, and the resulting build output is accessible through a network share. Multiple build types are supported, enabling developers and configuration managers to control how the project is built.

Since much of Team Foundation Server is written using the Microsoft® .NET Framework and Web services, it is easy to customize by subscribing to events and using the Team Foundation object model to automate tasks. I'll show an example of this that enables continuous integration by responding to source control check-in events.

Project Unit Testing

As an example, I'm creating, testing, and building a simple .NET library following the principle of extreme programming. I'm developing a math component that provides basic arithmetic functions using generics in the .NET Framework 2.0. A generic interface defines methods for the arithmetic functions such as add and subtract. This allows the Calculator library to invoke the correct methods based on type. The example library is not that important to the concepts conveyed, but can be explored to show the use of .NET generics. Note that all of the code for this article is available in both C# and Visual Basic®.

As all good extreme programming developers do, I first create unit tests for each of the public methods that I intend to implement. Typically, I stub out the public properties and methods before constructing the unit test. The new Class Diagram feature in Visual Studio makes it much faster to do this (see Figure 1).

Figure 1** CalculatorLibrary Class Diagram **

Unit tests are created for the public methods of the CalculatorLibrary class. With Visual Studio Team System, unit tests are added to the solution by creating a new unit test project, which allows you to create Web page tests, load tests, code unit tests, and even document tests that can be run manually. In this case a code unit test class helps test accuracy of the mathematic methods. Figure 2 shows an example of a unit test that makes sure the sum of all items added to the list equals 86. If an exception is encountered or the value is anything other than 86, the test will fail.

Figure 2 Unit Test for the Sum Method

'''<summary>Initialize the test</summary> <TestInitialize()> _ Public Sub Initialize() calculatorLibrary = New CalculatorLibrary(Of Integer)( _ New Math.Integer.Calculator()) End Sub ''' <summary> ''' Test Sum Method for Int type. (11+2+73=86) ''' </summary> <TestMethod()> _ Public Sub TestIntSumMethod() Dim itemsToAdd As List(Of Integer) = New List(Of Integer) itemsToAdd.Add(11) itemsToAdd.Add(2) itemsToAdd.Add(73) Dim result As Integer = calculatorLibrary.Sum(itemsToAdd) Assert.AreEqual(86, result) End Sub

The Initialize method defines any work necessary to set up the tests. In this case, the Calculator object is created. Test initialization methods must be decorated with the TestInitialize attribute. Likewise, test methods are decorated with the TestMethod attribute. These attributes are necessary for the methods to be recognized by the testing framework in Visual Studio, and later on when I build the product.

Figure 3** Reviewing Local Test Results **

Once unit tests are completed, the Calculator library can be implemented. Then I'm ready to run the unit tests locally. The Test Manager allows selection and execution of all tests located in the current solution. The Test Results window in Figure 3 shows that all my tests have passed and I'm ready to continue with the next step; specifically, creating a new team project in Team Foundation Server and adding the solution to it.

Creating a Team Project

After Team Foundation Server has been set up on the network, connecting to the server in Visual Studio is straightforward. In the Tools menu, just select Connect to Team Foundation Server. There is an option to add a new server, typically through a URL, to the server running on the default port of 8080. In my case, the URL, is TeamFoundation:8080.

When you are connected to the server, you can create a new project. Usually the name of the project is more encompassing than the name of the solution. I chose to call my project MSDN Calculator, as shown in Figure 4. Typically, a team project will be created before any code is written, as the team is going through the definition phase for the application being built. The team project might contain many solutions and Visual Studio projects, but the more interesting thing is that it's about much more than source control. Team projects have members with specific roles, they follow a specific software development methodology, they have a set of work item types corresponding to their methodology, they have a portal for collaborating and communicating projectrelated information, and so on. As such, a more appropriate team project name might be something like "Enterprise Infrastructure," as companies frequently have a set of common classes shared by many applications and need a team project to manage their development. However, I've gone with MSDN Calculator for simplicity of example.

Figure 4** Creating a New Team Project **

The next series of steps allow me to define the project in more detail. First I can define the process template that corresponds to the development methodology I'm using. As of Beta 3 the choices include Microsoft Solutions Framework (MSF) for Agile Software Development and MSF for CMMI Process Improvement. Since I'm using an extreme programming methodology, I'll choose the agile development template. Note, however, that these templates can be customized and you can create new templates as well. These templates are important because they define the work items, reports, and security settings for the project.

The next step is setting up source control for the project. The interface for doing this is much like Visual SourceSafe and will be intuitive to users who are migrating from that environment. For this project I accept the default to create an empty source control folder named after the project.

After completing this step, a dialog box confirms the settings I made and a new project is created. This takes some time because much work goes into creating a new project. On the back end, a SharePoint site is created, work items are generated, reports are defined, and all other setting are configured according to the process template. At this time, I can configure security on the project by defining roles for developers, testers, and other contributors. Now I can check in some items and define my build settings. In this case, I can also extend Team Foundation Server to support continuous integration.

Thus far I have created a .NET library, complete with a suite of unit tests, and I have created a new team project. The next step is to prepare the units tests so that they can run remotely on the Team Foundation Build Server.

Creating the Build Unit Tests

Within Visual Studio Team System, the Test Manager will automatically create a test metadata file with a .vsmdi extension. This file contains the unit test configuration for the project, which enables a unit test to be run with a remote build.

Within Test Manager I create two lists of tests by right-clicking in the left pane and selecting New Test List. I create one list that houses the unit tests for passing Ints to CalculatorLibrary and another for the tests that cover passing the Doubles (see Figure 5). Note that I did not include any of the manual tests since the build will be happening in the background with no human interaction. This data is saved in the .vsmdi file, which is checked into source control.

Figure 5** Project Test Configuration in Test Manager **

Within Team Foundation Server is the ability to define source control check-in policies, but I have chosen not to configure that for this project. A check-in policy is a great feature for imposing uniformity across a team of developers.

Figure 6** Project Contents **

At this point, the solution and project settings are configured as shown in Figure 6. This shows the main project that contains the Calculator library and the unit testing project. The test metadata file is also shown as a solution item that is bound to source control.

Defining the Build

I'm now ready to define build types for this project. In this case I am creating a build that will compile both the math library and the unit test project as debug builds and then run both of the test lists that I have defined. I name the build Continuous Integration. This name will be used later when I customize the build. If the build or any of the unit tests fail, I can notify the developers in various ways to correct the problem.

Within the Team Explorer window, the Team Builds option allows you to create new build types. The wizard takes you through the process to name the build type, select the projects to compile, and specify the build machine. In my configuration, I have the build service installed and running on the Team Foundation Server, but that will probably not be the typical scenario since the build is likely to be done on a standalone server. The last page of the wizard enables me to select the unit tests that the build will run after the project has compiled. In this case, I select both sets of unit tests that are defined (see Figure 7).

Figure 7** Selecting Tests to Run After a Build **

Once the Continuous Integration build is defined, the best way to test it is to force a build through the Visual Studio Team Explorer user interface. The Continuous Integration build can be seen under the Team Build folder within Team Explorer. A simple right-click will show the option to build the team project. Build status is reported within Visual Studio.

Extending Team Foundation Server

Team Foundation Server was built with extensibility in mind. One of the first indications of this feature that you're likely to notice is the ability to subscribe to events, particularly source control events. A command-line utility called BisSubscribe.exe is included with the Team Foundation Sever installation and is used to subscribe to these types of events. But before the subscription is set up, a Web service is created to catch the events. In Beta 3, the BisSubscribe utility can be found in the C:\Program Files\Microsoft Visual Studio 2005 Team Foundation Server\TF Setup folder.

BisSubscribe.exe allows you to define the types of events that you'll subscribe to and defines the exact signature and attributes of the Web service method used to catch the events. A proxy generated for the Notify Web method and modified to execute some work is shown here:

<SoapDocumentMethod(Action:="https://schemas.microsoft.com/ TeamFoundation/2005/06/Services/Notification/02/Notify", _ RequestNamespace:="https://schemas.microsoft.com/TeamFoundation/2005/06/ Services/Notification/02"), WebMethod()> _ Public Sub Notify(ByVal eventXml As String) ' implement event functionality ThreadPool.QueueUserWorkItem(AddressOf BuildProject, eventXml) End Sub

To create the subscription, you would run this command:

BisSubscribe /eventType CheckinEvent /userId PICCOLO\TFSSetup /address TeamFoundation:8080/Continuous/Build.asmx /deliveryType Soap /domain TeamFoundation

The address parameter is worthy of note. This is the location of the Web service that contains the Notify Web method. The easiest location is to create an additional Virtual Root beneath the Web site that contains the Team Foundation Server Web services (this is because the account that the Team Foundation Server application pool runs as already has permission to read the source and kick off a build; if you set up a different Web service, you'll need to ensure that it runs with credentials to do everything that needs to happen during a build). The physical location of these Web services is in the C:\Program Files\Microsoft Visual Studio 2005 Team Foundation Server\Web Services folder. As seen by the command-line options, I have added a virtual root called Continuous to service these requests.

The Notify Web method accepts an XML document in the eventXml parameter. This document contains all of the metadata for a particular event. Figure 8 shows the check-in event XML where I have checked in an intentional error in MathLibrary.cs.

Figure 8 Check-In Event XML

<CheckinEvent xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema"> <Artifacts> <Artifact xsi:type="ClientArtifact" ArtifactType="Changeset" ServerItem=""> <Url>https://TeamFoundation:8080//VersionControl/Changeset.aspx? artifactMoniker=90&webView=true</Url> </Artifact> <Artifact xsi:type="ClientArtifact" ArtifactType="VersionedItem" Item="MathLibrary.cs" Folder="$/MSDN Calculator/Msdn.MathLibrary/Msdn.MathLibrary" TeamProject="MSDN Calculator" ItemRevision="111" ChangeType="edit" ServerItem="$/MSDN Calculator/Msdn.MathLibrary/ Msdn.MathLibrary/MathLibrary.cs"> </Artifact> </Artifacts> <Title>MSDN Calculator Check-in 111: Added intentional error</Title> <ContentTitle>Check-in 111: Added intentional error</ContentTitle> <Owner>PICCOLO\\bwaldron</Owner> <Committer>PICCOLO\\bwaldron</Committer> <Number>90</Number> <CreationDate>10/15/2005 6:06:01 PM</CreationDate> <Comment>Added intentional error</Comment> <TimeZone>Pacific Daylight Time</TimeZone> <TimeZoneOffset>-07:00:00</TimeZoneOffset> <TeamProject>MSDN Calculator</TeamProject> <PolicyOverrideComment /> </CheckinEvent>

From the Notify Web method, I call another method that will build the project and report results to the development team. In ref="fig9">Figure 9, I extract the needed information from the event XML. Creating a build requires two steps. First, I need to create the parameters that are passed to the build controller to start a build. The minimum parameters are shown in the example. The build controller needs the Team Foundation Server name, project name, build server name, and the directory on the build machine to use for compilation. A predefined build type must be specified as well. In this case I supply the Continuous Integration build type that was defined previously in the team project.

Figure 9 Starting a Continuous Integration Build

Private Sub BuildProject(ByVal state As Object) Dim eventXml As String = CType(state, String) ' parse info from Event XML Dim xmlDocument As XmlDocument = New XmlDocument() xmlDocument.LoadXml(eventXml) Dim teamProject As String = _ xmlDocument.DocumentElement("TeamProject").InnerText Dim requestor As String = _ xmlDocument.DocumentElement("Committer").InnerText Dim changeTitle As String = _ xmlDocument.DocumentElement("ContentTitle").InnerText Dim offset As Integer = requestor.IndexOf("\"c) If offset > -1 Then requestor = requestor.Substring(offset + 1) End If ' create the build parameters for StartBuild method Dim buildParameters As _ Microsoft.TeamFoundation.Build.Proxy.BuildParameters = _ New Microsoft.TeamFoundation.Build.Proxy.BuildParameters() buildParameters.TeamFoundationServer = FoundationServer buildParameters.TeamProject = teamProject buildParameters.BuildDirectory = BuildDirectoryPath buildParameters.BuildMachine = BuildServer buildParameters.BuildType = BuildType Dim status As BuildConstants.BuildStatusIconID = _ BuildConstants.BuildStatusIconID.Default Try ' create the build controller, start build and record Dim buildController As BuildController = _ BuildProxyUtilities.GetBuildControllerProxy(FoundationServer) Dim uri As String = buildController.StartBuild(buildParameters) Dim buildData As _ Microsoft.TeamFoundation.Build.Proxy.BuildData = Nothing Dim buildStore As BuildStore = _ BuildProxyUtilities.GetBuildStoreProxy(FoundationServer) ' wait until the build completes While status = BuildConstants.BuildStatusIconID.Default 'wait a second to check status Thread.Sleep(1000) buildData = buildStore.GetBuildDetails(uri) status = CType(buildData.BuildStatusId, _ BuildConstants.BuildStatusIconID) End While If status = BuildConstants.BuildStatusIconID.BuildFailed Then ' create a workitem for the developer who checked in AssignWorkItem(requestor, changeTitle, buildData) End If NotifyGroup(requestor, buildData) Catch soapEx As SoapException ' TODO: e-mail build administrator End Try End Sub

After the build parameters are defined, I obtain a proxy to the Build Controller. This is a Web service proxy that calls the StartBuild method with the defined parameters. A unique Uniform Resource Identifier (URI) is returned from the StartBuild method that will allow me to check the status of the build. After starting the build, another proxy is created to connect to the Build Store, the object where I check the status of the build and its details. I use a while loop to contact the Build Store periodically, passing in the URI from the StartBuild method, in order to determine whether the build has completed.

Assigning a Work Item

I check the status of the build after it completes. If the build has failed, I call the AssignWorkItem method to assign a case to the developer who checked in the offending code. To do this I retrieve another Web service proxy, this time to the Work Item Store. This allows me to create and save a new work item, as shown in the code in Figure 10.

Figure 10 Assign a New Work Item

Private Sub AssignWorkItem(ByVal requestor As String, _ ByVal changeTitle As String, ByVal buildData As _ Microsoft.TeamFoundation.Build.Proxy.BuildData) Dim server As TeamFoundationServer = _ TeamFoundationServerFactory.GetServer(FoundationServer) Dim store As WorkItemStore = _ CType(server.GetService(GetType(WorkItemStore)), WorkItemStore) Dim workItemTypes As WorkItemTypeCollection = _ store.Projects(buildData.TeamProject).WorkItemTypes ' Enter the work item as a bug Dim workItemType As WorkItemType = workItemTypes("bug") Dim workItem As WorkItem = New WorkItem(workItemType) workItem.Title = String.Format("Build {0} broken because " & _ "check-in changes. Investigate and correct: {1} ", _ buildData.BuildNumber, changeTitle) workItem.Fields("System.AssignedTo").Value = requestor workItem.Fields("Microsoft.VSTS.Common.Priority").Value = 1 workItem.Save() End Sub

The work item will show up as a task within the My Work Items list for the developer who caused the build to fail. The work item assigned based on my intentional failure is shown in Figure 11. This task gives the details necessary for the developer to fix the error, and also tracks when the issue is completed. Many metrics can be derived from these work items, including how many times a build was broken during an iteration and the amount of time it took to fix the error. All of this information gives transparency to the development effort.

Figure 11** Automatically Assigned Work Item **

In addition to assigning a work item, an e-mail is sent out to the development team alerting them of the build status. An e-mail can be sent out even if the build is successful. A good extension point would be to integrate the ability to send an instant message or another alerting mechanism. An example of the e-mail for a failed build is shown in Figure 12.

Figure 12** Automated E-Mail Sent to Development Team **

The e-mail provides a link to the drop of the newly built project and to the build log. The build log contains information needed to determine the cause of a failed build or a failed test case. This entire workflow is automated without a single person having to check on the status or manually assign work items, enabling the development team to concentrate on the most important item: building quality software.

Conclusion

Visual Studio 2005 Team System and Team Foundation Server have many features that support agile development methodologies. With unit testing and work items ingrained into the product, using a TDD approach has never been easier within Visual Studio. As I've demonstrated, extending Team Foundation Server to support practices like continuous integration is as easy as catching events and connecting to the server's infrastructure.

Ben Waldron is a founding partner at Pollinate Media, an Interactive Branding and Innovation Company located in Portland, Oregon. Ben can be contacted for questions or opportunities via e-mail: ben.waldron@pollinatemedia.com.