Software-Development Methodologies and Visual Studio Team System
From Documents to Automation
Microsoft Visual Studio Team System
Microsoft SQL Server 2005
Summary: There is a diverse set of methodologies for different types of software-development life cycles. To implement these methodologies effectively and consistently, it is important to have life-cycle tools that automate the processes and artifacts of the methodologies. Microsoft Visual Studio Team System (VSTS) provides a compelling solution for methodology management and automation. (12 printed pages)
Automating a Life-Cycle Methodology in VSTS
Defining a New Process Template
Analyzing Typical Life-Cycle Methodologies
Creating Process Templates for Life-Cycle Methodologies
Enabling Process Implementation Beyond Visual Studio: APPRISE
Software-development life-cycle methodologies provide the what (processes and deliverables), how (techniques), and who (roles) for every typical role in a software-development project, such as solution architects, business consultants, and developers. Over the years, the software industry has matured to a great extent in terms of the methodologies used. Today, the industry has quite a number of established and proven software-development methodologies that cater to different kinds of project life cycles: Microsoft Solution Framework (MSF), Rational Unified Process (RUP), Waterfall, Agile, and so forth.
To use these methodologies effectively, it is important to follow the processes defined in the methodologies in a consistent manner across all projects. Following the software-development processes in a consistent and accurate manner is a challenging task because:
- Software-development processes are complex, involving many levels of interdependent activities. Most of the processes and methodologies are available as reference documents only. It is difficult for practitioners to learn the exact processes from the document references and follow them in their projects.
- Practitioners follow the processes manually and submit the required data into various life-cycle tools that do not have much integration with the engineering tools. The life-cycle tools could be isolated tools for project management, requirement management, bug tracking, or review management, whereas the engineering tools could be separate tools for activities such as designing, coding, testing, or Integrated Development Environments (IDE) such as Visual Studio. These different sets of tools make it difficult to have a consistent methodology implementation across multiple projects, and generate inconsistent data and reports in the organization.
Inconsistency makes the process analysis unreliable, thus making it difficult to identify the process improvements. Also, as the processes are followed manually, data collection and reporting become manual tasks, which results in lower productivity and delivery efficiency.
These problems can be addressed to a large extent by automating the implementation of methodologies using the life-cycle tools provided by different vendors like Microsoft Corporation and Borland. Microsoft offers Visual Studio Team System (VSTS), an integrated environment that covers most of the activities involved in a software-development project, including life-cycle management and automation.
VSTS implements its life-cycle methodologies by using process templates. (This article does not describe the internal structure of progress templates. If you are unfamiliar with that, I recommend that you refer to Process Template Plug-Ins on MSDN.) A process template is a set of XML files that provides specifications for different artifacts and processes of a methodology. To automate an organization's adopted life-cycle methodology using VSTS, you must define a process template for that methodology.
This article provides the required guidance for automating your established software-development methodologies using VSTS. Also, it illustrates the iterative process to create a new process template and describes the common components that constitute most of the available methodologies. Finally, it describes the different ways in which you can implement those components by using process-template elements.
VSTS uses a logical concept, team project, that represents a software-development project configured in the Team Foundation Server, or TFS. (VSTS uses the TFS component to provide a central storage for all project-related data and enable collaboration within the team members of the project. This server-side aspect of VSTS integrates several key concepts.) To obtain the benefits provided by VSTS, you must create a team project for every software-development project you start. To apply a particular methodology to the software-development project, you must configure its team project and populate the team project with initial data, as per that methodology. This initial configuration and data can be specified using process templates.
Process templates, as mentioned earlier, are a set of XML files with a predefined schema and structure. The Project Creation Wizard (PCW), which is used to create team projects, understands these XML files and their structure. While creating a team project, PCW reads the XML files from the process template, and configures and populates the team project accordingly. After the team project is created, the tools provided in VSTS (such as Work-Item Tracking, Source-Code Control, and so forth) automatically take care of running the project according to that configuration and data. Hence, to implement a custom life-cycle methodology in VSTS, the only thing you must do is define a process template for that methodology and create a team project using it. Configuration, execution, monitoring, and reporting are automatically handled by VSTS tools, as they understand the process-template schemas.
Defining a new process template is a complex task. You must plan carefully and perform a comprehensive analysis of your organization's existing processes and VSTS out-of-the-box process templates. You must identify the artifacts and processes in the existing methodology, and select the appropriate elements of VSTS process templates that you will use to implement those artifacts and processes. An artifact or a process can be implemented using more than one type of process-template elements. You must choose the one that fits best in the context of your existing processes. Creating a new process template is an iterative process. Generally, you can use the steps shown in Figure 1 to create a process template that meets your requirements. Figure 1 describes each step in detail.
Figure 1. The iterative process for creating a new process template (Click on the picture for a larger image)
Choose the Base Process Template
You should choose an existing process template as your base point and customize it per your requirements, instead of building a template from scratch. You can choose from either of the two out-of-the-box process templates:
- MSF for Agile Software Development
- MSF for Capability Maturity Model Integration (CMMI) Process Improvement
Alternatively, you can use a process template you defined previously for another methodology. To choose the right process template, you should analyze the methodology and break it down into the required processes and artifacts. Select the template that provides the most of the processes and artifacts required, so that customization can be kept to a minimum.
For example, if your methodology requires a Review process, it is better to use the MSF for CMMI Process Improvement template, because it has a Review Work Item Type (WIT). As another example, both of the templates have a Bug WIT, but if you only need a very simple process for bug tracking, you would choose the MSF for Agile Software Development template, which has a simpler "Bug" WIT.
After you have selected a base process template, you must customize it to suit your methodology. To do this, you must understand each artifact of your methodology and create a corresponding element in the process template, either by changing an existing XML file or creating a new XML file.
The first time you customize the process template, make only a small change. If you try to make a lot of changes, you might make multiple mistakes that will be difficult to debug. Plan your change, and update the appropriate XML to implement the change.
You must upload a process template before it can be selected for creating new team projects. Upload the customized process template to a TFS using the Process Template Manager provided in Team Explorer. It is recommended that you use a TFS that is not in use by other teams, because uploading a process template makes many changes in the Team Foundation data and schema—for example, new field definitions and their corresponding additions in the reporting warehouse—that are not easily reversible. You should upload a process template to a production TFS only when you have tested the template thoroughly and do not plan to make further changes to it.
Make sure that the name of your process template does not exist in TFS; you cannot upload two process templates with the same name. If you want to use a name that already exists in TFS, you must delete the existing process template.
The upload process performs a check to ensure that the XML is valid, as per the schemas defined in TFS. If the XML files in your process template are not valid or not compliant with the TFS schema, you will get errors, and the upload process will fail. In that case, find the changes you made, and identify the one that is causing the error. It is recommended that you do not make many changes in a single iteration, because it will make any problems more difficult to find.
Create Team Project
After uploading a process template, create a new team project using the PCW, and use the new process template to verify your changes. If your project creation fails due to errors, look at the project-creation log file (the last page of the PCW has a link to the log file). It contains lists of tasks that it performed and error details. You can map the failed tasks back to the XML files and determine the cause of the errors.
Even when a process template is successfully uploaded, it is possible for it to cause errors during the project creation. The upload process performs only a schema verification, whereas the project creation deals with a host of other systems: SQL Server databases, Source Control, Windows SharePoint Services, SQL Server Reporting Services, and so on. If there is a change that is not compliant with the data schemas or rules of these external systems, or if these systems are not accessible for some reason, you will get an error during the project creation.
Also, you do not have to create a new team project to test the changes in WITs. You can use the witimport command-line utility to upload an updated WIT into an existing team project and test the changes in the same team project. For more information, see the witimport reference on MSDN.
After the team project is created successfully, verify that your change appears in the Team Project correctly by opening the Team Project in Team Explorer. If you added a work-item type, make sure it appears in the Add New Work Item menu. If you removed an RDL file, ensure that the corresponding report is no longer available. You must perform a verification for each and every plug-in.
As explained in the previous section, the first step for process-template creation is to understand the various components or artifacts of the methodology to be implemented. There are quite a few methodologies available, representing various software-development life cycles. These methodologies generally have the following artifacts:
- Phases, iterations, or stages. For example, Waterfall methodology defines phases such as Requirement Engineering, Design, Coding and Unit Testing, and so forth.
- Ongoing processes that are common to all the phases or iterations of that life cycle—for example, Change Management, Risk Management, and so forth.
- Metrics and reports.
- Supporting documents (process descriptions, templates, checklists, and samples).
A typical phase is described by the following artifacts:
- Activities—Roles and workflows, processes
- Inputs—Entry criteria
- Outputs—Exit criteria
Let's go through each of the key artifacts from the typical methodologies, in order to understand its implementations in the process templates. Please note that one artifact can be implemented in more than one way using different elements of a process template, and different people can choose from different approaches to implement the same process or artifact.
Because process templates are a set of XML files, to implement an artifact you must either modify one of the existing XML files or create a new XML file. Describing the exact XML syntax for an artifact implementation is beyond the scope of this article. However, the article would describe how to identify the appropriate process-template elements and how to use them for implementing an artifact. To find the exact XML syntax required, see the process-templates documentation available on MSDN.
Phases/Stages and Activities
Many life cycles maintain a three-level hierarchy, where the life cycle is broken into phases containing activities that represent the types of tasks to be performed, and where activities contain the actual tasks. The life cycles also require many reports to be generated at the end of each phase. These requirements can be best met by representing phases and activities using the iteration hierarchy in the VSTS Classifications Plug-in. The VSTS hierarchies support multiple levels. You can maintain as many levels as required in the methodology you are implementing. For example, for a phase-activity-task relationship, you can keep two levels: the first level for phases and the second level for activities.
All the work-item types have fields to select the level of classification hierarchies to which they belong. By representing phases and activities using the iteration hierarchy, any task (or work item) can be associated to an activity (and the phase containing that activity) by choosing that activity in the work item's iteration field.
Figure 2 shows a task being associated with an activity and a phase (for a waterfall life cycle) through Microsoft Project.
Figure 2. Associating a task to an activity and phase (Click on the picture for a larger image)
Also, because these hierarchies are predefined dimensions in the VSTS data warehouse, it is very easy to generate phase- or activity-based reports.
The previous solution is sufficient, as long as you need only the "name" of the phase and its associated work items. Generally, however, you will have to store additional attributes about the phase, such as the start date, end date, state, and so forth. However, the classification hierarchies do not support any attributes other than the name. In this case, it is best to define a "Phase" work-item type, which can store all the required attributes. You would have to define one work item (in addition to the iteration hierarchy) for each of the phases in that project. This can be done easily by having a predefined work item for each phase in the work-item plug-in files of the process template.
To associate a work item with a hierarchy, you can create only one WIT, "Phase," for a particular phase, and set the work item's iteration path to the hierarchy node corresponding to that phase. However, there is no well-designed solution to restrict Team Foundation users from creating more than one work item of one particular type, as all the work-item types get displayed in the Add New Work Item menu. The best possible solution is to educate users not to create additional work items of that type. To assist the users, you can name the work item in such a way that the users understand that they should not create work items of that type. For example, you can name a work-item type for the phase like "[For Internal Use] Phase" or "[Do Not Create] Phase." Also, you can customize the WIT in such a way that no one has permissions to create the work item, except for the users in a particular role. You can do that by restricting the permissions for the state transition for the initial state to that particular role. For example, the following statement allows only "TFS Administrators" to change the state to "New."
<TRANSITION from="" to="New" for="TFS Administrators"></TRANSITION>
Forms, Workflows, and Processes
Activities are generally related to roles. For example, developers write code and perform reviews, whereas testers perform testing and raise bugs. These activities can involve one or more processes that must be followed to perform those activities. For example, Review can be performed using a simple Peer Review process or a more detailed Fagan Inspection process. Each of these processes might have the following:
- Fields and rules governing their possible values. For example, a Peer Review process requires fields such as Work Product Being Reviewed, Reviewer, Work Product Size, and so on.
- Workflow states. For example a bug-tracking process requires that a bug should have states such as New, Active, Resolved, or Closed.
- Possible workflow-state transitions. For example, a bug state can be changed from Active to Resolved or the obverse, but it cannot be changed from Closed to Active.
- Workflow-state transitions rules and actions. For example, there could be a rule such that when a bug is in the Closed state, it cannot be assigned to anyone, or when a bug is closed by someone, the Closed Date and Closed By fields are populated automatically.
- Form layout that defines the look-and-feel and positioning of various fields of that process.
All the previously mentioned requirements can be easily defined by creating a WIT for that process. For example, to define the Peer Review process, you will create an XML file for the Peer Review WIT that becomes part of your process template. When you create a team project based on that template, you can create the Peer Review work items based on that WIT, which enable you to comply automatically with the Peer Review process. Figure 3 shows a Peer Review work item in the waterfall life cycle inside a Visual Studio–based VSTS client.
Figure 3. Peer Review work-item user interface
There are also a few things you cannot do with the Work Items framework. For example, you do not have a rule that specifies that the value of one field is always more or less than that of another field (required for the Start Date and Finish Date fields). Also, you do not have rules for specifying the minimum and maximum values. For example, you might want to restrict the Percentage field between 1 and 100. Also, there is no straightforward method to specify that a user can write to one type of work item, and not to another type. However, you can find workarounds for most of these limitations.
Sometimes, life-cycle methodologies also provide predefined artifacts that all projects implementing that methodology should have. For example, tasks like "Set Up the Development Environment" and "Create Traceability Matrix" are required in all the development projects. Similarly, work products like Design Document and System Test Plan are also required for most projects. Having such artifacts helps project managers to a great extent, as they do not have to take care of defining the common activities or tasks, because these come automatically from the methodology. You can also create predefined artifacts for methodologies implemented in VSTS. You can do this by defining the work-item instances in the XML file for the work-item tracking plug-in.
Version Control-Related Processes
Version control is an activity that is fundamental to any software-development project. Most organizations have well-defined processes for version control of their source code and other items that require version control, such as documents, binaries, and so on. VSTS provides a new enterprise-class version-control system in TFS. You can implement your existing version-control processes using a version-control plug-in. Using this plug-in file, you can specify different types of check-in notes, general policies (for example, whether exclusive check-out is allowed or not), and permissions (the activities that different roles can perform).
VSTS also provides check-in policies that enable custom rules such as All Unit Tests Should Pass Before Check-Ins. However, these are client-specific policies and cannot be enabled through process templates.
Entry and Exit Criteria
Generally, each phase or stage has some entry and exit criteria. These can be implemented in one of the following ways:
- Have the field Is Exit Criteria? in all the work-item types that can represent the exit criteria. If the value for that field is "Yes" for a work item, it means that the work item is one of the exit criteria for the phase to which the work item belongs. For example, if "Document SRS" is a task in the Requirement Engineering phase and has the field Is Exit Criteria? as "Yes," it means that this task is one of the exit criteria for the Requirement Engineering" phase.
You can save a Work Item Query (using the Team Explorer query builder) for each phase, which finds all the work items in that phase with the exit criteria of "Yes."
The same thing also applies to the Entry Criteria. The out-of-the-box templates in VSTS follow this approach:
- You can define a special type of WIT, which represents exit criteria. Generally, methodologies have a distinct artifact for deliverables or work products, which also act as the exit criteria. Such artifacts can be best represented by a WIT for them. In such cases, all the work items for that type become exit criteria for the phase with which they are associated.
For enforcement purposes, if you want to distinguish between mandatory and non-mandatory exit criteria, you can have an Is Mandatory field in the WIT.
Metrics and Reports
Life-cycle methodologies define metrics related to different artifacts and aspects of the project, to help you evaluate the project performance and comply with set quality standards. These metrics must be tracked and delivered as reports to help you make decisions. There are two steps involved in getting a report for a particular metric:
- Ensure that all the data required for the report is available in the reporting database.
- Create a report that does the necessary calculations and formats the data, as required.
VSTS maintains a SQL Server 2005–based OLAP data warehouse to keep the data for all its reports. The data from different subsystems of the VSTS (such as work-item tracking and classifications) updates the cubes and dimensions in this reporting data warehouse at regular intervals. You can then write reports with the SQL Server 2005 Reporting Services, which are deployed in the team projects you create.
WITs give you the capability to specify the data that should go to the reporting warehouse. You can add a "reportable" attribute to the fields of WITs. All the fields marked as reportable get pushed to the warehouse as either a dimension or a measure, according to what you specify in the field definition. Once you have the required data in the warehouse, you can define reports for a methodology by creating RDL files and adding them to the Reporting Plug-in of the process template.
Supporting Documents: Process Descriptions, Templates, Checklists, and Samples
Usually, the processes and the related documents (templates, checklists, samples, and so forth) are available on the organization's quality Web site. Because the site contains many processes and documents, it becomes very difficult for a developer to look for something specific. You can use VSTS in two ways to simplify this:
- Specify all the documents related to a methodology in the XML files for the WSS Plug-in. When a team project is created with that methodology, all these documents are added to the specified document library in the WSS site. You can access those documents through either the WSS site or Team Explorer. Team Explorer allows you to access all the document libraries in the associated Team Project.
- Create process-guidance pages (used by the WSS Plug-in) for your methodology that would give very intuitive and connected Web pages for developers to browse through, to understand the different processes and activities.
VSTS process templates provide six out-of-the-box plug-ins that can address most requirements of your methodologies. However, there could be instances in which these six plug-ins do not meet your requirements. For example, your organization's quality department might maintain a central database that contains organization goals or thresholds for various metrics (such as effort variance and schedule slippage) that every project should meet. You can implement this by fetching the goal values from this central database at the time of project creation, and populating them in custom work-item types in VSTS. Currently, none of the available plug-ins can meet this requirement. In such cases, you can write your own plug-in using the extensibility API.
In the previous sections, we covered most of the artifacts of a life-cycle methodology. The mappings between methodology artifacts and VSTS process templates are summarized in Table 1.
Table 1. Mapping of Hewlett-Packard GM artifacts and VSTS process-template elements
|Methodology artifacts||VSTS process-template elements|
|Phases and activities||First-level and second-level nodes, respectively, in iteration hierarchy of Classifications Plug-in|
|Roles and permissions||Groups and Permissions Plug-in|
|Workflows and processes (tasks, bugs, reviews, issues, risks, change requests)||Work-item types of Work-Item Tracking Plug-in|
|Predefined artifacts (tasks, milestones)||Predefined work items in Work-Item Tracking Plug-in|
|Version control-related rules and permissions||Version-Control Plug-in|
|Entry and exit criteria (inputs and outputs)||Work-item types with a field for specifying if a particular work item is an exit or entry criteria, and work-item queries for getting those work items|
|Measures and metrics (for reports)||Fields marked as reportable in related work-item types|
|Supporting documents||List of all the required documents under WSS Plug-in|
There are a few limitations of the VSTS methodology-management system. The two key ones are:
- No aggregation across Team Projects. This means that you are not able to get reports showing data across projects in drill-up, drill-down fashion.
- Bias towards Visual Studio projects. In other words, every developer has to install Visual Studio Team Explorer, even if they are working on a non-Visual Studio project (for example, Java-based projects).
However, these limitations are applicable to out-of-the-box VSTS features. You can address these issues by extending VSTS using the Object Model and the Web Services exposed by VSTS. In fact, VSTS provides a broad range of extensible points that you can use to develop your own solutions—offering many more features on top of VSTS. One example of such a solution is APPRISE (Adaptive Project and Process Regulation and Information System for Enterprise), which is discussed in the next section.
For large organizations that already have well-established processes and life cycles and are also compliant with quality standards like CMMI, the out-of-the-box features of VSTS might not be sufficient to fulfill all the requirements. Such organizations need a complete solution for managing their projects and processes that can serve all kinds of projects, not only Visual Studio–based projects. Interestingly, the extensibility features of VSTS make it a good candidate to use as a platform for building such a solution.
Hewlett-Packard has developed such a solution—called APPRISE—which is built over VSTS Team Foundation Server and provides many additional features, such as a Web-based client, organization structure support, implementation of HP Global Methods (GM) methodologies, process enforcement, a schedule-tracker service, and aggregated metrics and reports. To get more details about APPRISE, see the case study.
There is a diverse set of methodologies for different types of software-development life cycles. To implement these methodologies effectively and consistently, it is important to have life-cycle tools that automate the processes and artifacts of the methodologies. Microsoft VSTS is one such system, and provides a compelling solution for methodology management and automation.
VSTS implements a methodology using process templates. You can "customize" out-of-the-box process templates to suit your methodology requirements. However, there are certain functionalities, such as report aggregation, that you cannot get with "customization" alone. To achieve this, you can "extend" VSTS by using its extensible points.
About the author
Sanjay Narang is a Senior Technology Consultant at Global Delivery India Center (GDIC) of Hewlett-Packard. He is based in Bangalore, India, and has about eight years of IT experience. Sanjay has done Post Graduate Diploma in IT (PGDIT) and holds an MCSD for .NET Framework and SCJP certifications. He has been involved in designing and developing solutions based on Microsoft technologies for a long time. Also, he has worked on various solutions and products related to software-process automation and software-quality standards, such as SEI CMMi.
Apart from software-process automation and VSTS, Sanjay is working on SOA-related projects around Microsoft technologies and their interoperability with other technologies. He has published papers around IPv6, VSTS, SOA, and Web Services, and writes extensively about VSTS in his blog. Sanjay can be contacted at firstname.lastname@example.org.