Export (0) Print
Expand All
Expand Minimize
5 out of 5 rated this helpful - Rate this topic

Sarbanes-Oxley 404 and Visual Studio Team System 2008

Visual Studio 2008

Andrew Delin, Microsoft Corporation

April 2008

Microsoft Visual Studio Team System 2008 can be used to assist a business in collecting information that concerns its software-development practices, which may be of use towards compliance with a regulatory framework, such as Sarbanes-Oxley section 404 ("Sarbanes-Oxley 404") internal-control verification and testing. (30 printed pages)

Visual Studio Team System 2008

Microsoft Visual Studio Team System 2008 can be used to assist a business in collecting information that concerns its software-development practices, which may be of use towards compliance with a regulatory framework, such as Sarbanes-Oxley section 404 ("Sarbanes-Oxley 404") internal-control verification and testing.

Visual Studio Team System 2008 is not a general-purpose compliance package. The Sarbanes-Oxley Act of 2002—and, in particular, Sarbanes-Oxley 404—has a broader scope than software-application development, and Visual Studio Team System 2008 is useful in supporting only a subset of the total compliance activities that a business must undertake. Each business has its own unique combination of Sarbanes-Oxley 404 risks and controls; the scenarios in this article must be considered as examples, instead of as being required for a customer's compliance reporting.

This article provides example scenarios for using Visual Studio Team System 2008 to assist you in supporting corporate compliance with the Sarbanes-Oxley Act of 2002 when you develop software. Sarbanes-Oxley Act compliance is an example of a regulatory framework that spans the enterprise and requires a consistent management approach. The concepts that are described here might apply to the use of Visual Studio Team System 2008 with other frameworks, such as the Control Objectives for Information and related Technology (COBIT), the Committee of Sponsoring Organizations of the Treadway Commission (COSO), and the International Organization for Standardization (ISO).

Visual Studio Team System 2008 is a collection of tightly integrated software-development tools that are designed to help teams be more productive in producing software. Visual Studio Team System 2008 tracks the detailed activities of the software-development team, and can assist in supporting compliance with Sarbanes-Oxley 404 requirements as they relate to software development.

In a compliance environment, Visual Studio Team System 2008:

  1. Can act as a collector of detailed project information, including decisions that are made, artifacts that are produced and actions that are taken.
  2. Can be used to produce at any time ad-hoc reports that detail the past or present state of the project.
  3. Could be useful in the presence of a formal risk-control framework that is defined by a Sarbanes-Oxley 404 expert.
Cc441754.note(en-us,VS.90).gifNote:
This article describes some example scenarios that demonstrate how Visual Studio Team System 2008 could be applied to support the implementation of Sarbanes-Oxley 404 requirements in an organization, in the context of software development. These scenarios are for illustration only, and should not be considered "standard" or "recommended" Sarbanes-Oxley 404 controls. Each organization would develop its own compliance framework—capturing different data and defining different processes—to meet its regulatory obligations.

Visual Studio Team System 2008 can be used to assist a business in collecting information that concerns its software-development practices, which may be of use towards compliance with a regulatory framework, such as Sarbanes-Oxley section 404 ("Sarbanes-Oxley 404") internal-control verification and testing.

Visual Studio Team System 2008 is not a general-purpose compliance package. The Sarbanes-Oxley Act of 2002—and, in particular, Sarbanes-Oxley 404—has a broader scope than software-application development, and Visual Studio Team System 2008 is useful in supporting only a subset of the total compliance activities that a business must undertake. Each business has its own unique combination of Sarbanes-Oxley 404 risks and controls; the scenarios in this article must be considered as examples, instead of as being required for a customer's compliance reporting.

This article provides example scenarios for using Visual Studio Team System 2008 to assist you in supporting corporate compliance with the Sarbanes-Oxley Act of 2002 when you develop software. Sarbanes-Oxley Act compliance is an example of a regulatory framework that spans the enterprise and requires a consistent management approach. The concepts that are described here might apply to the use of Visual Studio Team System 2008 with other frameworks, such as the Control Objectives for Information and related Technology (COBIT), the Committee of Sponsoring Organizations of the Treadway Commission (COSO), and the International Organization for Standardization (ISO).

Visual Studio Team System 2008 is a collection of tightly integrated software-development tools that are designed to help teams be more productive in producing software. Visual Studio Team System 2008 tracks the detailed activities of the software-development team, and can assist in supporting compliance with Sarbanes-Oxley 404 requirements as they relate to software development.

In a compliance environment, Visual Studio Team System 2008:

  1. Can act as a collector of detailed project information, including decisions that are made, artifacts that are produced and actions that are taken.
  2. Can be used to produce at any time ad-hoc reports that detail the past or present state of the project.
  3. Could be useful in the presence of a formal risk-control framework that is defined by a Sarbanes-Oxley 404 expert.
Cc441754.note(en-us,VS.90).gifNote:
This article describes some example scenarios that demonstrate how Visual Studio Team System 2008 could be applied to support the implementation of Sarbanes-Oxley 404 requirements in an organization, in the context of software development. These scenarios are for illustration only, and should not be considered "standard" or "recommended" Sarbanes-Oxley 404 controls. Each organization would develop its own compliance framework—capturing different data and defining different processes—to meet its regulatory obligations.

The Sarbanes-Oxley Act of 2002 was signed into law on July 30, 2002, in response to corporate governance failures within the United States. Given that many U.S. firms have business activities and partnerships in Asia, Africa, Europe, and Australasia, the Act can be said to have global consequences. Sarbanes-Oxley 404 mandates certification that attests to the effectiveness of internal control over financial reporting (ICFR). Certification aside, implementation of the requirements of the Act can help an organization achieve process improvement and performance targets.

The examples in this article apply to operational controls that could be used within the area of software-application development, which represents one area from a larger suite of Sarbanes-Oxley 404 internal controls for a company. This article illustrates how the MSF for CMMI® Process Improvement process template in Visual Studio Team System might be used in the generation and collation of information to support internal controls that might be required to be produced at audit and testing stages. Using Visual Studio Team System 2008 as described here could assist the auditee in sourcing information that is required during the auditing process, as well as being able to produce the requisite information on an ad-hoc basis.

Cc441754.note(en-us,VS.90).gifNote:
The Software Engineering Institute Capability Maturity Model® Integration (CMMI®) is a benchmark model for software development from Carnegie Mellon University, represented in a process template for Visual Studio Team System 2008. Special permission to use SEI and its materials ©2005 by Carnegie Mellon University, in MSF for CMMI Process Improvement is granted by the Software Engineering Institute. CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

Compliance with a regulatory framework is sometimes viewed as an onerous undertaking—placing demands on people, and consuming resources from many parts of the organization. Fundamentally, however, compliance is about risk management—protecting the owners and stakeholders of a business from risks that might arise because of incomplete or missing business processes. When these erroneous outcomes affect suppliers, partners, or consumers, significant financial impacts can result, as well as harm to image and reputation. Achieving compliance is a demonstration that defined regulatory objectives from across the organization are being achieved—not just when the auditor visits, but throughout the business year. Depending on the quality of the framework that is in place, compliance can equate to greater confidence for senior management and business stakeholders that "the right thing is being done"—and being able to demonstrate this—instead of having CXOs and business decision makers admit, "I didn't know this was happening." The law has little accommodation for the latter position.

This article uses a number of terms that have specific meanings, as they relate to corporate compliance. This section introduces some of this terminology:

  • Framework—An overall collection of defined risks and associated mitigation activities (or "controls") that span the business and have a specific compliance focus, such as corporate financial reporting.
  • Risk-based—The framework is based on an up-front, clearly defined risk appraisal in the context of Sarbanes-Oxley 404. The risk appraisal defines a set of specific financially related risks that become the focus of control objectives (as described later).
  • Performed in accordance to a formally defined framework—The framework has a substantial degree of rigor: It is usually defined by a qualified, authorized appraiser and customized for each business; it contains checkpoints and tests; and senior managers must attest to the resulting data that appears on corporate financial reports, and so on.
  • Evidence-based—The framework is substantive, that is, it generates evidence that actions are occurring within the business to demonstrate the management of identified risks. Without such evidence, controls cannot be demonstrated as working.
    Cc441754.note(en-us,VS.90).gifNote:
    When we use the term "evidence" in this article in connection with Visual Studio Team System 2008, we generally mean "information," and do not mean to imply that information that is generated by Visual Studio Team System 2008 is sufficient to demonstrate compliance. You should work with your own counsel and Sarbanes-Oxley 404 experts to determine that question.
  • Continuous—The actions within the framework have frequency and continuity, which means that it does not "happen" just at the time of audit; it is an ongoing process. Auditors will typically attempt to establish that the processes that are described are happening on an ongoing basis, and will drill into ad-hoc samples to confirm this.
  • Action-oriented—Compliance is not achieved by a tool, but by defined actions that are performed on a regular basis by people (for example, the project board must periodically review risks and document their decisions against each item). It is a characteristic of successfully implemented frameworks that tool support and human activity combine to deliver the level of compliance that is required. Frequently, activities flow across departmental boundaries, between personnel, and between systems—requiring a coordinated approach to achieve compliance.
  • Customer-related—A customer (or business owner) is deemed to be any group—either internal or external—for whom contracted software-application development work is performed, where this work is deemed of significance for auditing purposes to Sarbanes-Oxley 404 requirements.

Sarbanes-Oxley 404 requires that an annual report be produced concerning the effectiveness of ICFR. This must be attested to by the company's auditors as being an accurate assessment by management of the firm. To prepare for this, a company must show that it has put in place internal controls that can withstand audit testing to demonstrate that the controls work. Creating and validating these controls is typically a time-consuming exercise. These controls exist within a framework of risks, which lead to the establishment of controls, which define activities and produce evidence, which can be used as feedback, to demonstrate management of the risk and improvement of the process.

An example of this framework is given in the next section.

After the framework is in place, the production of the annual Sarbanes-Oxley 404 report requires that data be produced for a set of defined controls for a business. The scenarios that appear later in this article outline the activities around a set of example controls for software-application development—showing how Visual Studio Team System 2008 contributes towards the production and reporting of evidence for these controls. For each of these scenarios, data would be produced in support of the production of the annual report for Sarbanes-Oxley 404 certification.

While there is commonality among many businesses—such as order processing, accounts payable, manufacturing, R&D, distribution, sales and marketing, and so on—it is nonetheless necessary for a business to carry out an individual risk assessment for its own enterprise. The purpose for the business is to establish a set of specific risks that are relevant to Sarbanes-Oxley 404, and then to establish procedures that manage these risks successfully over time. Figure 1 shows the importance of the initial risk assessment and how the management of each specific risk relates to this:

Cc441754.c2c4ebd3-efbb-4c00-96ae-4ae2988f0f6b(en-us,VS.90).png

Figure 1. Regulatory framework for this business

The initial risk assessment is made by the business, in conjunction with its compliance expert. The scope of this assessment is broad—potentially, touching many areas across the firm. This might be a phased exercise, with the more significant business areas and risks being established first.

For each applicable risk that is identified by the business, a regulatory control objective is written that declares the intention of the risk management. For example, the specific risk might be that "the customer/business owner is not charged for changes that are made to the software," and the corresponding control objective would be "to guarantee that all chargeable changes to software are delivered for billing." A control activity is then put in place; this is a procedure with steps that fulfill this control objective—for example, to prepare a report regularly that indicates the billable status (yes or no) of every change to the system. The control activity also stipulates that the report be forwarded to the Accounting department for harmonization with its billing records. The reports and the records of their transmission to Accounting form the evidence for this control.

Because regulatory frameworks span a wide range of business activities beyond the IT domain, Visual Studio Team System 2008 should be expected only to assist partially in the implementation of a regulatory framework within an organization. Visual Studio Team System 2008 is not a general-purpose compliance package; however, the following sections will demonstrate that it can aid in the production and collation of evidence that is required at the audit and testing stages of a compliance review or audit.

Figure 2 indicates the extent to which Visual Studio Team System 2008 can support the total Sarbanes-Oxley 404 compliance requirements of an organization:

Cc441754.266a8717-c092-44ce-90b8-cc61e765ee45(en-us,VS.90).png

Figure 2. Whole of business compliance (complete set of control objectives)

It is important to note that Sarbanes-Oxley 404 specifically relates to transactions that affect assets. In organizations in which software development has a significant financial impact, Visual Studio Team System 2008 can assist a business to capture and report evidence towards further demonstrating its compliance to Sarbanes-Oxley 404 requirements. While Sarbanes-Oxley 404 focuses on financial controls, other frameworks—such as COBIT, COSO, and ISO—have more general objectives, such as quality or industry-standards compliance. Visual Studio Team System 2008 also can assist in the capture and reporting of evidence in support of these other frameworks—using similar customizations, as outlined in the remainder of this article.

In addition, while Team System might be helpful in gathering information about the software-development process and producing a cumulative record of the development process, Visual Studio Team System 2008 does not embed rights-management functionality into such a cumulative record. Accordingly, in consultation with its Sarbanes-Oxley 404 expert, a business might want to consider enhancing the reliability of the records that are maintained/generated by Visual Studio Team System 2008 by, for example, including procedural controls (such as formal approval and recording) and education to project administrators and team members—advising them of the significance of the use of certain tools that might change or delete data, and what impact this might have on the overall compliance regime, and so on.

This article presents a set of brief scenarios that indicate how Visual Studio Team System 2008 might be used to support some example controls. A reasonable degree of familiarity with the CMMI template for Visual Studio Team System 2008, and work items in general, is assumed. For our purposes, we will assume that the business under question has worked with a Sarbanes-Oxley 404 expert appraiser to develop a comprehensive risk analysis of its business.

As is typical for such an assessment, the resulting register might contain between 50 and 300 risks across the enterprise, for which controls will need to be implemented. Within this list are six risks that relate to software-application development in the IT department of this firm. These possible risk scenarios are as follows:

Table 1. Sarbanes-Oxley 404: Risk scenarios for application development

1.

Significant project risks are not identified or mitigated when they arise during the project.

2.

Unauthorized changes are incorporated into the project.

3.

The status of customer change requests is not tracked.

4.

There is a failure to understand or define customer requirements.

5.

The customer/business owner is not charged for changes to the system.

6.

Changes are not incorporated into the released system.

The following sections will examine the IT-related risk scenarios that are listed and will discuss their implementation, including Visual Studio Team System 2008. Extending these scenarios to cover wider Visual Studio Team System 2008 features (for example, source-code branching to manage releases) is outside of the scope of this article.

Scenario 1: Effective Management of Significant Project Risks

This scenario examines the specific area of the adequacy of risk management on software projects in the organization. If risk management on a software-development project is inadequate in practice, significant unchecked risks could lead to cost implications for the business and direct financial impacts on external stakeholders.

Item

Detail

Identified risk

Significant project risks are not identified and mitigated, as and when they arise during the life of the project.

Area

Risk management.

Control objective

Ensure that all significant project risks are identified and mitigated during the life cycle of the project.

Control activity

A risk register is maintained within Visual Studio Team System 2008 for the life of the project. The project manager ensures that risks are entered into Visual Studio Team System 2008, with any escalated risks being reviewed at the Project Risk Board.

"Evidence"

Risk records, Escalation reports and Risk-tracking records, minutes of the Project Risk Board.

Implementation Using Visual Studio Team System 2008

This scenario can be implemented in Visual Studio Team System 2008 by defining a process for the management of significant risks and adding a small number of additional fields to the CMMI Risk work item (see Fields Used, later in this section). In this context, we will define "significant" as any Risk work item that has a Severity of Critical or High. ("Severity" is an existing field.)

Outline of Required Process

The standard CMMI template for Visual Studio Team System 2008 includes a Risk work item. By adding the fields that are described later in this section, we are able to use the CMMI template to support the concept of a "significant risk register" for compliance purposes.

When a significant risk that is not easily closed arises in a project, it would be marked as "Escalated," and comments would be added to the History field. This allows risks to be reviewed by the Project Risk Review Board (or similar)—being a formally defined group that will review each significant risk for probable impact, and manage a plan for mitigation and contingency. The notes that are taken at these review meetings will be recorded in the (existing) History field in each Risk work item.

While the Iteration Path (existing field) could be used to infer in which iteration the risk is to be resolved, the new Target Date field is a more explicit means to declare the expected date by which this risk will have been mitigated to a non-significant state. It is expected that the Target Date field would be set by the Accountable Risk Owner (new field).

The (existing) Closed By and Closed Date fields should be set by the Accountable Risk Owner. Closure of the Risk is not considered the same as "Resolved by"/"Resolved date," which is the state before final approval is given to mark the Risk as closed. Closure would be approved by the Project Risk Board during a regular review meeting for significant risks. You can alter the work-item state workflow so that the Resolved to Closed transition can be performed only by the Accountable Risk Owner—being a member of a new Accountable Risk Owners security group. (Given that risk owners might include nontechnical personnel, the Web access client might be appropriate to provide access to Visual Studio Team System 2008.)

The business will be able to create a Project Risk Charter document, which defines the risk-management approach to significant risks. This would include a list of Project Risk Board members, which would match the list of members within the Accountable Risk Owners security group. Any changes to the Project Risk Board membership would be reflected in this document, and the security group would be updated on the same day. The Project Risk Charter would indicate also when the Project Risk Board would meet. The Project Risk Charter document also would define "significant" as being the High or Critical value in the Risk Severity field in the CMMI template. The Project Risk Charter also would include details of where the Visual Studio Team System 2008 server was located, which Visual Studio Team System 2008 project names were overseen by the Board, and so on.

Notes or minutes that result from the Project Risk Board meetings could be a useful component of any compliance framework. These minutes may be attached to the Risk work item (using the existing Attachments field), or alternatively a resource pointer (URL or file reference) may be entered into the History field (such as a pointer to a document within Microsoft Office SharePoint). The minutes might document, for example, whether mitigation and contingency plans in the Risk work item have been reviewed and updated, or other ways in which risk is being managed actively.

The business will be able to use Team System to help track that significant project risks are being managed in a structured and accountable way by the use of queries, as in the following list:

  1. Produce a list of the risks with Severity of High and Critical.
  2. Examine each Risk by its State field, so that Active, Resolved, and Closed risks might all be examined.
  3. By using the Links field, link from each Risk in this list to the corresponding Task work items, to demonstrate project activity that takes place over time to address each risk. A query could be implemented in Office Excel to show the linked Tasks that derive from a parent Risk.
  4. Examine the History field of each Risk work item to reveal the actual changes to the Risk over time—satisfying the business that real activity was taking place during the life of the Risk; for example, we might examine what reasons were recorded in the History at the time that the Risk was marked as "Escalated."
  5. Compare the Resolved and Closed dates for Risks, to ensure that a Project Risk Board meeting took place between these (that is, not Resolved and Closed on the same day).
  6. Compare the Closed and Target dates, to examine the efficacy of risk-management timing.
  7. Compare the Closed By field to the formally documented Accountable Risk Owners security group.

The preceding list is not an exhaustive one, but it is a high-level example of what can be achieved with Visual Studio Team System 2008. It might be taken as a starting point for implementing a formal project risk register for software-application development projects.

You will see from this scenario that a combination of human activity and tool support is needed to support the aims of the regulatory framework for a business. The benefit of using Visual Studio Team System 2008 is in the proximity of the tool to the real daily activity on the project; the information that is captured is more likely to be reflective of what actually takes place, and tends to encourage better enactment of team processes (such as risk management) due to this close shadowing of team software development.

Fields Used

In addition to the existing fields in the Risk work item, the following fields may be added by the customization of the CMMI template:

New field

Notes

Accountable Risk Owner

Field that represents the name of the person who is responsible for the life of the risk and its end result; this name must be a member of a new Accountable Risk Owners security group.

Escalated

Flag field that indicates whether this risk has been escalated to the Project Risk Board for review. All risks for which this field is set affirmatively will be reviewed at a regular Project Risk Board meeting.

Target Date

Field that identifies specifically a target date by which the risk should have been closed.

Example

This section illustrates some views of the Visual Studio Team System 2008 client, and shows a path to implement the Risk methodology that was described earlier. These views are possible after the customizations to the Risk work item have been made in the CMMI template.

Figure 3 shows a query that is performed to examine project Risks. The three additional fields that are added to the Risk work item—Accountable Risk Owner, Escalated, and Target Date—appear highlighted in yellow:

Cc441754.a4a38186-2da7-483d-af30-bac817ea4f61(en-us,VS.90).gif

Figure 3. Query examining project risks

This query produces a list of eight Risks that are classified in Severity as either Critical or High. The first item in the list is highlighted, and the work-item form is shown below the list. Each Risk has an Accountable Risk Owner who takes formal responsibility for seeing it through to completion, along with a Target date for resolution as described in the preceding sections (see Outline of Required Process).

The query can provide any set of relevant fields, with specific filters to limit the records that are displayed. Queries may be built readily in Visual Studio Team Explorer; this delivers a benefit in being able to create queries rapidly for auditing purposes.

Figure 4 shows the definition of a query that is used to show the "SOX 404 Escalated Risks":

Cc441754.544aaec0-cc48-4772-80da-de753863fd71(en-us,VS.90).gif

Figure 4. Definition of query used for showing "SOX 404 Escalated Risks"

The preceding image shows the criteria for selecting the aforementioned Risks. In this case, the user has elected to view only Risk work items that are either Critical or High in Severity, and that have a State of Active, Resolved, or Closed. In addition, the user is interested only in Risks that have been marked as Escalated to the Project Risk Board, so the user includes this in the query definition.

The fields that are shown in the results list may be configured separately from the query definition itself. This allows additional data to be shown for sorting purposes.

Figure 5 shows the dialog box for choosing the fields for the query result set:

Cc441754.dd5a6d65-7176-4227-a472-a753f8c19a51(en-us,VS.90).gif

Figure 5. Column Options dialog box, for choosing fields for query result set

Scenario 2: Prevention of Unauthorized Changes to the Project

This scenario explores the level of rigor that is associated with the process of making changes to the software that is under development. When changes are not managed closely, discrepancies can arise over delivered functionality when compared to that which was contracted. When changes cannot be accounted for clearly, these can have serious implications—for example, in project scheduling, security, and destabilization or interference with existing functionality.

Item

Detail

Identified risk

Unauthorized changes are incorporated into the project.

Area

Change management.

Control objective

Ensure that all requests for project changes are authorized through formal procedures and that they do not compromise the project.

Control activity

Change Requests are approved according to the Project Change Board terms of reference (a written document). All Change Requests require that an impact and risk assessment be performed. The Change Board is responsible for ensuring that the appropriate impact and risk analyses have been completed as part of their change review and approval process. Relevant information is included in every Change Request document.

"Evidence"

Approved Change Requests, and copies of Project Change Board minutes that show review of Change Requests.

Implementation Using Visual Studio Team System 2008

The existing CMMI template already includes a Change Request work item. With the addition of some extra fields (see Fields Used, later in this section) and changes to current fields, it is possible to offer a change-request system with an increased level of provability.

Outline of Required Process

Central to a formal Change Request process are the ideas of impact analysis and risk assessment. The CMMI template includes several "Impact on" fields; by making these mandatory, we will now capture a statement of the impact of the change from various perspectives (architecture, test, user experience, and so on). In addition, a new "Impact on Scheduling" field requires a statement of the project-management implications that this change brings.

As per the risk scenario that is described in Scenario 1, each Change Request work item must have an Accountable Change Owner field. This person is responsible for ensuring that the impact and risk assessments are undertaken, and also has a vital role in moving the work item from the Proposed state to the Active state and, later, from the Resolved state to the Closed state. The former transition (Proposed -> Active) is important, because it represents an agreement that the change will proceed and that it has been properly appraised and understood for impact; the latter transition (Resolved -> Closed) is important, because it reflects that the change has been verified as being correctly carried out, according to the details in the Change Request, and that the impact and risk have been managed. (To support further the correct use of these state changes, one could add specific REASON values to the TRANSITIONS within the Change Request work item XML definition—for example, "Proposed Change Agreed" and "Change Verified Correct" in the preceding state changes.)

The state changes may be made only by a member of the Accountable Change Owners security group. Custom alert e-mail messages can be configured to e-mail these people or a discussion list when the state changes to one of interest. If these state changes are not explicit enough, it is possible to imagine the use of an additional custom field, such as "Customer Approval Given By name" (see Scenario 4).

A Change Board could be established for the purposes of reviewing all Change Requests routinely. The operation of this group of people would be defined in a Change Board terms-of-reference document. Within the terms of reference could be included the frequency of meeting and also the membership—with the same persons being listed in the new Accountable Change Owners security group. This document could be kept in synch with the security group that controls the fields and states for the Change Request work item.

The implementation of the change may be scheduled to take place in a specific iteration during the development of the software. The Iteration Path field could be used to capture to which iteration the change is allocated.

Before a change can proceed (that is, move into the Active state), an impact analysis and risk assessment may be performed. At the end of this, one would expect the impact fields to be populated and to see one or more Risk work items linked to the Change Request work item. Each of the linked Risks would describe a risk to be managed in implementing this change. The Risk work items represent the risk assessment and risk-management activity. The existing Links field would be used to create these associations.

The business can prepare a list of all Change Requests for the period that is under examination, grouped by Proposed, Active, Resolved, and Closed. For those Change Requests that have progressed beyond the Proposed stage, the business can review the notes or minutes from the Change Board meeting that discussed each change. These minutes can be attached to the Change Request work item (by using the existing Attachments field); or, alternatively, a resource pointer (URL or file reference) can be entered into the History field (such as a pointer to a document within Office SharePoint). The minutes would indicate that the Impact fields were reviewed, and that the risk assessment (the set of Risk work items) was approved.

The Change Request can be seen as instantiating a new requirement in the system. This allows one to demonstrate what new work was created and when it was undertaken, in implementing the change. This can be done by linking a Requirement work item with this Active Change Request, which can show the new Task work items that implemented that Requirement, including the dates on which they were performed. This approach is also consistent with the cumulative flow reporting that is used by the team and/or project managers.

As seen earlier, several queries could be used to obtain information that the change-request process is taking place in a disciplined manner—for example, a list that shows:

  1. Change Requests in their Proposed, Active, Resolved, and Closed states.
  2. The Change Request title and the linked child Task work items—each with owners, dates, and status.
  3. Change Requests in Resolved (but not Closed) state, for the purpose of showing which Change Requests have not been approved for closure by the Change Board.
  4. The Impact fields—containing, for example, some substantial description that captures the team's thinking, instead of just a tacit indication that the impact was considered.
  5. Risk work items that are associated with the Change Request.

Fields Used

The following fields can be added to the Change Request work item by customization, to enable more detailed tracking of changes in a formally defined change-request process:

New field

Notes

Accountable Change Owner

Name of the person who is both the proposer and closer of changes to the software that is under development; this person is responsible for providing the impact assessment and risk analysis for the change, and for approving its final closure in accordance with the details in the Change Request. The Accountable Change Owner must be a member of a new Accountable Change Owners security group.

Impact on Scheduling

Field that extends the existing set of Impact on fields in the CMMI template by adding a space for recording the impact on scheduling that this Change Request will cause when it is implemented. This forms part of the impact assessment.

Impact on ArchitectureImpact on TestImpact on Design/DevelopmentImpact on Tech PubsImpact on User Experience

Existing CMMI template fields each of which should become mandatory (required), so that commentary must be keyed explicitly into all of them (even if No impact is entered).

Example

Figure 6 shows a modified Change Request work item to capture the extended data that is associated with a formally defined change process, as described in the preceding scenario.

Cc441754.note(en-us,VS.90).gifNote:
To support the Change Request scenarios that are described in this document, the following additional fields have been added: Accountable Change Owner, Change Is Billable, Customer Requested Change, and Customer Approval Given By. These fields have been added to a new "Compliance" group on the work-item form. In addition, a new Impact on Scheduling field has been added to the top of the Analysis tab. (All new fields appear highlighted in yellow.)
Cc441754.a9e97434-22da-4407-a8f3-cf013acf20a1(en-us,VS.90).gif

Figure 6. Change Request work item, modified

Figure 7 shows the results of a query to display Change Requests as a starting point for examining Change Requests and their related Task work items:

Cc441754.c26a3435-3432-4a68-ade0-901972c622dc(en-us,VS.90).gif

Figure 7. Query results displaying Change Requests for examining Task work items

To view the related Task work items, each Change Request must be opened, and its Links tab must be examined, as shown in Figure 8:

Cc441754.0634bf59-069e-4c5b-b6d9-54dcdbc47030(en-us,VS.90).gif

Figure 8. Review of linked work items for Change Request 202

Figure 9 shows a query that may be used to retrieve the Change Requests. This image shows the query being modified to select only Change Requests that are in a Resolved state in the workflow—representing those items that are still to be approved for closure by the Change Board (see the preceding point 3):

Cc441754.6a41c352-0359-486d-8862-e9e1070a3103(en-us,VS.90).gif

Figure 9. Query modified to select only Change Requests in Resolved state

Scenario 3: Appropriate Tracking of Change Requests

This scenario looks at the tracking of Change Requests from inception to closure. To avoid the risk of changes not being tracked properly, a sufficiently structured process is needed so that the requisite set of work that is associated with each change is captured and executed. Customers have the right to ask for an accurate status of progress against their approved change and to have an early indication of any alteration to schedule, overrun of time, or reduction in functionality.

Item

Detail

Identified risk

Status of Change Requests is not tracked.

Area

Change management.

Control objective

Ensure that project changes are tracked and status is updated on a regular basis.

Control activity

All changes are entered into Visual Studio Team System 2008 upon creation, and their progress is tracked through approval to closure.

"Evidence"

Change Request records.

Implementation Using Visual Studio Team System 2008

This scenario extends the preceding one (Scenario 2) by looking specifically at the area of the tracking of Change Requests and exploring the relationship of work items that represent a change to the system that is under development.

Outline of Required Process

You can see the life history of a real change that takes a Change Request from Proposed to Resolved, and the relationship of this change to the original Requirement. This can be achieved by one or more queries that show the Change Requests for each Requirement and, below this, the set of Task work items that hang from each Change Request, as shown in Figure 10:

Cc441754.54d511e7-b872-41c5-9cc4-23f243090799(en-us,VS.90).png

Figure 10. "Life history" of Change Request, with related Requirement and Tasks

This structure uses links between work items to indicate that a Requirement is implemented by Tasks, a Change Request belongs to a Requirement, and the Change Request is implemented by Tasks. This is achieved by specifying "Work Item" when adding the link type between items, as shown in Figure 11:

Cc441754.e794a53d-5cdc-4050-b7d3-1561c7a5f412(en-us,VS.90).gif

Figure 11. Add Link dialog box, showing addition of link type

This assumes that the Project Change Board (from Scenario 2) has decided to use Change Requests to "parent" all of the work that is to be done for each change to the system. This decision should be documented in the Change Board charter document that outlines how Change Requests are managed.

The business also might want to track a range of Tasks over time, between the dates when the Change Request was Approved and Resolved; these Tasks should have appropriate information in their History fields.

To explore further the aim of tracking project changes on a regular basis, a query can be run to highlight Change Requests in the Resolved or Closed state. These should be expected to have one or more related, completed Task work items that represent the work that is done (see the following example). If these conditions are not the case, further examination will be required to find out what degree of tracking and recording is being done after Change Requests are entered into Visual Studio Team System 2008.

Fields Used

This scenario can be implemented by using the existing fields in the Change Request work item. Other work items also are employed.

Example

From the preceding description, Figure 12 shows a query for Change Requests that have moved beyond the Proposed state:

Cc441754.2d86e739-ee27-4a22-b3f0-11f112f40887(en-us,VS.90).gif

Figure 12. Query for progressed Change Results

To see the corresponding "parent" Requirement and subordinate development Tasks, the Change Request must be opened and the Links tab examined, as shown in Figure 13:

Cc441754.68fe65c2-00e8-4d1b-992d-c8599d2a4c40(en-us,VS.90).gif

Figure 13. Links tab open for examination in Change Request

From this point, each related work item in the list can be checked to ensure that the state corresponds to the Change Request; for example, one would expect a closed Change Request to have Closed development Tasks and the "parent" Requirement at least to be in the Active state, if not the Resolved or Closed state.

Scenario 4: Customer Changes Are Defined and Understood

This scenario explores the customer's involvement in the change process. The customer must be satisfied that the team has understood and engaged with the change being requested. A customer's request for a change incurs cost: inadequate or incorrect delivery has a range of potential financial and other impacts on the customer's business.

Item

Detail

Identified risk

Failure to understand or define customer requirements.

Area

Change management.

Control objective

Ensure that all customer-originating changes to requirements are authorized by the customer, prior to change implementation.

Control activity

All changes to requirements that are requested by the customer require a Change Request. The Change Request will move through the change process in a documented manner. The customer will form an additional approval level for the change.

"Evidence"

Approved Change Requests, copies of Change Board minutes, customer-approval artifacts that are attached to the Change Request (for example, e-mail).

Implementation Using Visual Studio Team System 2008

This scenario can be implemented by adding some additional fields to the Change Request work item (see Fields Used, later in this section) and requiring the attachment of customer-approval evidence to the work item.

Outline of Required Process

Every change to the system requires that a Change Request be raised. The process of making a change is defined in the terms-of-reference document of the Change Board. This scenario requires that the customer becomes an additional approval point for the Change Request to proceed from Proposed to the Active state. This can only be done when the customer has reviewed the plan for the proposed change and has indicated positively that they accept the details of the change. This evidence would be stored in the Attachments field of the Change Request work item; in addition, the customer's name would be recorded in the new Customer Approval Given By field.

The History field is an existing field on each work-item type that is a cumulative log of all field changes to the work item, with time stamping (the History field supports querying of a work item "as of" a particular date—for example to show Change Request #501 on July 1). If a Change Request was approved by the customer, the History field will indicate when the value of this field changed (when an approving customer name was keyed). You might expect to see this change happen during the Proposed state or, at least, on the date when the Change Request work item became Active.

Attached evidence—such as an e-mail from the customer that refers to the specific change (for example, including a reference to the version number of a design document)—provides a means to confirm the customer's communication on a given date, regarding a change approval and the specific version that was agreed to.

The business might require a list of which Change Requests are customer-originated, as well as a list of the customer names. This can be provided by a query.

For integrity, the Change Request must link to the original Requirement work item, so that one can associate the customer's change with the originally requested feature of the system.

The business might require to see minutes of the Change Board to demonstrate formal discussion of this Change Request, including reference to the customer in the meeting; these minutes could be attached to the Change Request (Attachments field) or, alternatively, linked (Links field) to an intranet document library or file share (for example, the URL of a document in Office SharePoint).

Normally, only the customer (if a member of the team) or a member of the Change Board might be making the transition from Proposed to Active. A business can look for this by requesting a query for this data in the History field. As discussed in previous scenarios, this might be implemented technically through the use of a security group, such as Accountable Change Owners, if applicable.

Fields Used

Adding the following fields to the Change Request work item enables an extra level of functionality around customer approval for change:

New field

Notes

Customer Requested Change(Yes/No)

Field that indicates whether the change was initiated by the customer or another source. The scenario focuses on customer-sourced changes only, instead of an internal design change, and so on.

Customer Approval Given By (name)

Name of the customer who has approved the change. The presence of a name in this field indicates approval for the change to proceed. This field should be set on or before the date that the Change Request moves to the Active state. One might require that this field be populated before the work-item state can be changed from Proposed to Active.

Example

At present, it is not possible to search within the field-change log inside the History field. However, a query can be constructed to show all Change Requests that were requested by Customers (instead of internally generated) and that have been marked as approved by the Customer.

After one has such a list, a visual inspection of the History fields allows a business to determine when the Customer Approval field was completed in each case. In addition, one can inspect the History field to see when the Change Board minutes were added to the Change Request work item, and can read the attached minutes. Each change to the Work Item is recorded in the History field, along with the name of the user who made that change.

Figure 14 shows a query to show customer changes to the system, where the customer name is specified and the Change is Active, Resolved, or Closed:

Cc441754.2e4821c5-83d4-4c73-b2dc-b5dfb0b18667(en-us,VS.90).gif

Figure 14. Customer-change query

By opening a Change Request item from this query, one can see the following Attachments tab that shows the Change Board meeting minutes, which is attached as an Office Word document, as shown in Figure 15:

Cc441754.14d90010-ce3b-4b9a-8dc4-a5e3202d3f46(en-us,VS.90).gif

Figure 15. Attachments tab of Change Request item

By switching to the History tab, one can see the detail of when the Change Board minutes were attached to the Change Request, as well as the person who moved this Change Request from the Active state to the Resolved state. The data in the History field is cumulative and—if appropriate mechanisms are put in place—cannot be changed, so that it can assist in tracking the activities that take place on the Change Request, such as field changes, state changes, the attachment of documents, and so on, as shown in Figure 16:

Cc441754.a1dd75d7-370f-4eb6-b112-efb4814ed6aa(en-us,VS.90).gif

Figure 16. History tab of Change Request item

Scenario 5: Customer Is Charged Correctly for All Changes

This scenario examines conditions under which changes to software were not billed (that is, not correctly billed) to the customer.

Item

Detail

Identified risk

Customer/Business owner is not charged for change.

Area

Change management/Engagement management.

Control objective

Ensure that all closed, billable changes are delivered for billing accurately and completely.

Control activity

Identification of billable changes; frequency is per billable change.

"Evidence"

Derived from the reports that show which changes were billable.

Implementation Using Visual Studio Team System 2008

Visual Studio Team System 2008 is not a billing engine, but it can be used to deliver information about project changes to the accounting department for reconciliation purposes. Because Visual Studio Team System 2008 closely shadows the development process, it is a ready vehicle for capturing and documenting information about changes to the software that is under development, including the billable status of each changes and the amount of work that is performed in making each change.

Outline of Required Process

When changes are approved by the customer and/or the Change Board, the Change Is Billable field should be marked at the time when the Change Request moves from the Proposed state to the Active state. A degree of rigor is necessary to ensure that this field is set correctly. A business could see an attached piece of information from the customer that indicates their agreement about the billable status of this change. This would be done by using the Attachments field. The business can query for a list of all Change Requests that are marked as Billable (per new field, seen earlier) and can inspect the Attachments of these work items to confirm that the customer was aware that billing for these changes was to occur.

The business will be able to see evidence that billing is reconciled with accounting processes in the organization. A query might be requested that lists all Change Requests that are marked as billable having a Closed state or Resolved state—perhaps, including a unique customer-account identifier for accounting purposes (this could be an additional field in the Change Request work item). This list will be reconciled against the accounting department, in the expectation of finding complete consistency between the software-development processes in the company and the billing/charging processes to the customer.

The business will be able to see completed Task work items that are associated with the billable Change Request, to confirm that the volume of work that is described to the customer actually was undertaken.

Fields Used

This scenario can be implemented by the addition of one additional field to the Change Request work item in the CMMI template:

New field

Notes

Change Is Billable(Yes/No)

Field that indicates that this change is to be billed to the customer for implementation as an extension or alteration to existing agreed project charges.

Example

Figure 17 shows a query that could be used to locate Resolved and Closed Change Requests that are marked as billable to the customer:

Cc441754.62fcec0c-98e5-4236-9bea-5270a5543a1e(en-us,VS.90).gif

Figure 17. Query showing billable Change Requests, both Resolved and Closed

Examining the Links tab for Change Request number 202 reveals the linked Task work items that fulfilled this change to the system, as shown in Figure 18:

Cc441754.7a0987cd-0d10-4fe8-a015-8e27e1a8c316(en-us,VS.90).gif

Figure 18. Links tab of Change Request item

To determine the actual work that was done to perform the change (that is, the number of completed hours), each Task work item must be opened in turn. The work that was performed for Change Request number 202 should reconcile with a related billable charge to this customer in the accounting system.

Scenario 6: Customer Changes Appear in the Delivered System

This scenario looks at the risk that commissioned functionality is not present in the system that is delivered. This might have financial impacts: The contract can be demonstrated as being unfulfilled, and the customer's business might be affected if the system does not perform as expected.

Item

Detail

Identified risk

Change not incorporated into release.

Area

Change management/Release management.

Control objective

Ensure that all changes are implemented into a release.

Control activity

Changes will link to the requirements, so that the completed build has an audit trail from original requirements through deployment.

"Evidence"

A list of all requirements incorporated into a build, whether new or by change request. Acceptance test results, which demonstrate the implementation of the required change.

Implementation Using Visual Studio Team System 2008

This scenario can be achieved through the use of linked work items and ad-hoc queries to demonstrate requirements traceability through time. Visual Studio Team System 2008 is particularly helpful in tracking work in this manner, because of its close shadowing of the development process from requirements to product builds.

Outline of Required Process

The business will be able to see that approved changes that proceed to implementation subsequently appear in a release. The Change Request work item includes an Integrated In field that lists the build number in which the change is present. This number is present, so that testing can validate the Change Request.

When a Change Request is moved to the Active state, one or more specific tests will be written as coverage of the changed functionality. These tests will be linked to the Change Request work item, and a query can be used to show the testing that demonstrates the implemented change.

From a software testing point of view, all tests that are associated with a Change Request work item must indicate a pass condition. In addition, acceptance tests that include customer sign-off are presented as confirmation that the change was seen and approved in the release.

Fields Used

This scenario can be implemented by using the existing work-item fields.

Example

Figure 19 shows a query that lists the status of all Customer Approved Change Requests and shows into which build each Change Request was integrated:

Cc441754.46e962d7-baa3-4234-8c8c-da1c39404fbf(en-us,VS.90).gif

Figure 19. Query showing delivered changes

The resulting list reveals one closed Change Request that does not appear in a build for release (see work item 202 in Figure 20). From this point one might examine why a Closed, customer-approved Change Request did not appear in an integration build:

Cc441754.06c6107e-be85-4038-9998-1cbe7de17e64(en-us,VS.90).gif

Figure 20. Additional query results, showing integration build

The following fields could be added by customization of an existing template to allow the capturing and querying of evidence, as described. We recommend that these customizations be made to a CMMI-based template, because this includes a set of concepts that are relevant to a formalized software-development environment—such as the presence of an auditor function on the team and existing work-item types, such as Risk and Change Request. Nonetheless, these changes conceivably could be applied to an Agile or other template to support the same compliance aims (other customizations and additional Work Items might be required).

The following list shows the possible customizations in this article to support the provided scenarios:

New field

Work-item type

Notes

Accountable Risk Owner

Risk

Name of the person who is responsible for the life of the risk and its end result; in the provided scenarios, this name would be a member of the new Accountable Risk Owners security group.

Escalated

Risk

Flag field that indicates whether this risk has been escalated to the Project Risk Board for review. All risks that have this field set affirmatively would be reviewed at a regular Project Risk Board meeting.

Target Date

Risk

Field that identifies specifically a target date by which the risk should have been closed.

Accountable Change Owner

Change Request

Name of the person who is both the proposer and the closer of changes to the software that is under development; this person would be responsible for providing the impact assessment and risk analysis for the change and approving its final closure in accordance with the details in the Change Request. In the provided scenarios, the Accountable Change Owner would be a member of a new Accountable Change Owners security group.

Impact on Scheduling

Change Request

Field that extends the existing set of Impact on fields in the CMMI template by adding a space for recording the impact on scheduling that this Change Request will cause when it is implemented. This might form part of the impact assessment.

Impact on Architecture,Impact on Test,Impact on Design/Development,Impact on Tech Pubs,Impact on User Experience

Change Request

Existing CMMI template fields each of which could become mandatory (required), so that commentary would have to be keyed explicitly into all of them (even if No impact is entered).

Customer Requested Change(Yes/No)

Change Request

Field that indicates whether the change was initiated by the customer or another source. The scenario focuses on customer-sourced changes only, instead of an internal design change, and so on.

Customer Approval Given By (name)

Change Request

Name of the customer who has approved the change. The presence of a name in this field would indicate approval for the change to proceed. This field could be set on or before the date on which the Change Request moved to the Active state. One might require that this field be populated before the work-item state can be changed from Proposed to Active.

Change Is Billable(Yes/No)

Change Request

Field that indicates that this change is to be billed to the customer for implementation as an extension or alteration to existing agreed project charges.

In addition to these customizations, the article defines also two security groups (Accountable Risk Owners and Accountable Change Owners) and two project meetings (Project Risk Board and Project Change Board), along with suitable charters for the operation of Risk management and Change management.

Compliance is concerned with sound ongoing practice, and Visual Studio Team System 2008 can help in compliance environments. Its strong focus on team-process enactment means that the tool shadows the software-development action, from requirements to delivered builds. Visual Studio Team System 2008 can be used to retrieve detailed information about team actions over time, and to document the history and linkage of work that is undertaken. Used in the context of a set of controls that are designed to address specific risks for software-application development, Visual Studio Team System 2008 can support the process of complying with a regulatory framework, such as Sarbanes-Oxley section 404.

For the latest information about Visual Studio Team System 2008, see www.microsoft.com/teamsystem.

Andrew Delin is a product planner at Microsoft, specializing in team practices and methodologies including Agile and CMMI. Andrew has over twenty years of experience as a project manager, facilitator and trainer, helping teams to define vision, improve communication and deliver business value.

Did you find this helpful?
(1500 characters remaining)
Thank you for your feedback

Community Additions

ADD
Show:
© 2014 Microsoft. All rights reserved.