Sarbanes-Oxley 404 and Visual Studio Team System 2008

Switch View

Sarbanes-Oxley 404 and Visual Studi...

Collapse All

Andrew Delin, Microsoft Corporation

April 2008

Summary

Microsoft Visual Studio Team System 2008 can be used to assist a business in collecting information that concerns its software-development practices, which may be of use towards compliance with a regulatory framework, such as Sarbanes-Oxley section 404 ("Sarbanes-Oxley 404") internal-control verification and testing. (30 printed pages)

Applies to:

Visual Studio Team System 2008

Background

Value of Compliance

Compliance Terminology

Preparing for Sarbanes-Oxley 404 Certification

Outline of a Regulatory Framework in Action

Using Visual Studio Team System 2008 to Support Compliance

Introducing the Scenarios

Customizations to Implement These Controls in Visual Studio Team System 2008

Introduction

Visual Studio Team System 2008 is not a general-purpose compliance package. The Sarbanes-Oxley Act of 2002—and, in particular, Sarbanes-Oxley 404—has a broader scope than software-application development, and Visual Studio Team System 2008 is useful in supporting only a subset of the total compliance activities that a business must undertake. Each business has its own unique combination of Sarbanes-Oxley 404 risks and controls; the scenarios in this article must be considered as examples, instead of as being required for a customer's compliance reporting.

This article provides example scenarios for using Visual Studio Team System 2008 to assist you in supporting corporate compliance with the Sarbanes-Oxley Act of 2002 when you develop software. Sarbanes-Oxley Act compliance is an example of a regulatory framework that spans the enterprise and requires a consistent management approach. The concepts that are described here might apply to the use of Visual Studio Team System 2008 with other frameworks, such as the Control Objectives for Information and related Technology (COBIT), the Committee of Sponsoring Organizations of the Treadway Commission (COSO), and the International Organization for Standardization (ISO).

Visual Studio Team System 2008 is a collection of tightly integrated software-development tools that are designed to help teams be more productive in producing software. Visual Studio Team System 2008 tracks the detailed activities of the software-development team, and can assist in supporting compliance with Sarbanes-Oxley 404 requirements as they relate to software development.

In a compliance environment, Visual Studio Team System 2008:

Can act as a collector of detailed project information, including decisions that are made, artifacts that are produced and actions that are taken.
Can be used to produce at any time ad-hoc reports that detail the past or present state of the project.
Could be useful in the presence of a formal risk-control framework that is defined by a Sarbanes-Oxley 404 expert.

Note:
This article describes some example scenarios that demonstrate how Visual Studio Team System 2008 could be applied to support the implementation of Sarbanes-Oxley 404 requirements in an organization, in the context of software development. These scenarios are for illustration only, and should not be considered "standard" or "recommended" Sarbanes-Oxley 404 controls. Each organization would develop its own compliance framework—capturing different data and defining different processes—to meet its regulatory obligations.

Note:

This article describes some example scenarios that demonstrate how Visual Studio Team System 2008 could be applied to support the implementation of Sarbanes-Oxley 404 requirements in an organization, in the context of software development. These scenarios are for illustration only, and should not be considered "standard" or "recommended" Sarbanes-Oxley 404 controls. Each organization would develop its own compliance framework—capturing different data and defining different processes—to meet its regulatory obligations.

Visual Studio Team System 2008 can be used to assist a business in collecting information that concerns its software-development practices, which may be of use towards compliance with a regulatory framework, such as Sarbanes-Oxley section 404 ("Sarbanes-Oxley 404") internal-control verification and testing.

In a compliance environment, Visual Studio Team System 2008:

Can act as a collector of detailed project information, including decisions that are made, artifacts that are produced and actions that are taken.
Can be used to produce at any time ad-hoc reports that detail the past or present state of the project.
Could be useful in the presence of a formal risk-control framework that is defined by a Sarbanes-Oxley 404 expert.

Note:
This article describes some example scenarios that demonstrate how Visual Studio Team System 2008 could be applied to support the implementation of Sarbanes-Oxley 404 requirements in an organization, in the context of software development. These scenarios are for illustration only, and should not be considered "standard" or "recommended" Sarbanes-Oxley 404 controls. Each organization would develop its own compliance framework—capturing different data and defining different processes—to meet its regulatory obligations.

Note:

This article describes some example scenarios that demonstrate how Visual Studio Team System 2008 could be applied to support the implementation of Sarbanes-Oxley 404 requirements in an organization, in the context of software development. These scenarios are for illustration only, and should not be considered "standard" or "recommended" Sarbanes-Oxley 404 controls. Each organization would develop its own compliance framework—capturing different data and defining different processes—to meet its regulatory obligations.

Background

The Sarbanes-Oxley Act of 2002 was signed into law on July 30, 2002, in response to corporate governance failures within the United States. Given that many U.S. firms have business activities and partnerships in Asia, Africa, Europe, and Australasia, the Act can be said to have global consequences. Sarbanes-Oxley 404 mandates certification that attests to the effectiveness of internal control over financial reporting (ICFR). Certification aside, implementation of the requirements of the Act can help an organization achieve process improvement and performance targets.

The examples in this article apply to operational controls that could be used within the area of software-application development, which represents one area from a larger suite of Sarbanes-Oxley 404 internal controls for a company. This article illustrates how the MSF for CMMI® Process Improvement process template in Visual Studio Team System might be used in the generation and collation of information to support internal controls that might be required to be produced at audit and testing stages. Using Visual Studio Team System 2008 as described here could assist the auditee in sourcing information that is required during the auditing process, as well as being able to produce the requisite information on an ad-hoc basis.

Note:
The Software Engineering Institute Capability Maturity Model® Integration (CMMI®) is a benchmark model for software development from Carnegie Mellon University, represented in a process template for Visual Studio Team System 2008. Special permission to use SEI and its materials ©2005 by Carnegie Mellon University, in MSF for CMMI Process Improvement is granted by the Software Engineering Institute. CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

Note:

The Software Engineering Institute Capability Maturity Model® Integration (CMMI®) is a benchmark model for software development from Carnegie Mellon University, represented in a process template for Visual Studio Team System 2008. Special permission to use SEI and its materials ©2005 by Carnegie Mellon University, in MSF for CMMI Process Improvement is granted by the Software Engineering Institute. CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

Value of Compliance

Compliance with a regulatory framework is sometimes viewed as an onerous undertaking—placing demands on people, and consuming resources from many parts of the organization. Fundamentally, however, compliance is about risk management—protecting the owners and stakeholders of a business from risks that might arise because of incomplete or missing business processes. When these erroneous outcomes affect suppliers, partners, or consumers, significant financial impacts can result, as well as harm to image and reputation. Achieving compliance is a demonstration that defined regulatory objectives from across the organization are being achieved—not just when the auditor visits, but throughout the business year. Depending on the quality of the framework that is in place, compliance can equate to greater confidence for senior management and business stakeholders that "the right thing is being done"—and being able to demonstrate this—instead of having CXOs and business decision makers admit, "I didn't know this was happening." The law has little accommodation for the latter position.

Compliance Terminology

This article uses a number of terms that have specific meanings, as they relate to corporate compliance. This section introduces some of this terminology:

Framework—An overall collection of defined risks and associated mitigation activities (or "controls") that span the business and have a specific compliance focus, such as corporate financial reporting.
Risk-based—The framework is based on an up-front, clearly defined risk appraisal in the context of Sarbanes-Oxley 404. The risk appraisal defines a set of specific financially related risks that become the focus of control objectives (as described later).
Performed in accordance to a formally defined framework—The framework has a substantial degree of rigor: It is usually defined by a qualified, authorized appraiser and customized for each business; it contains checkpoints and tests; and senior managers must attest to the resulting data that appears on corporate financial reports, and so on.

Evidence-based—The framework is substantive, that is, it generates evidence that actions are occurring within the business to demonstrate the management of identified risks. Without such evidence, controls cannot be demonstrated as working.

Note:
When we use the term "evidence" in this article in connection with Visual Studio Team System 2008, we generally mean "information," and do not mean to imply that information that is generated by Visual Studio Team System 2008 is sufficient to demonstrate compliance. You should work with your own counsel and Sarbanes-Oxley 404 experts to determine that question.

Continuous—The actions within the framework have frequency and continuity, which means that it does not "happen" just at the time of audit; it is an ongoing process. Auditors will typically attempt to establish that the processes that are described are happening on an ongoing basis, and will drill into ad-hoc samples to confirm this.
Action-oriented—Compliance is not achieved by a tool, but by defined actions that are performed on a regular basis by people (for example, the project board must periodically review risks and document their decisions against each item). It is a characteristic of successfully implemented frameworks that tool support and human activity combine to deliver the level of compliance that is required. Frequently, activities flow across departmental boundaries, between personnel, and between systems—requiring a coordinated approach to achieve compliance.
Customer-related—A customer (or business owner) is deemed to be any group—either internal or external—for whom contracted software-application development work is performed, where this work is deemed of significance for auditing purposes to Sarbanes-Oxley 404 requirements.

Preparing for Sarbanes-Oxley 404 Certification

Sarbanes-Oxley 404 requires that an annual report be produced concerning the effectiveness of ICFR. This must be attested to by the company's auditors as being an accurate assessment by management of the firm. To prepare for this, a company must show that it has put in place internal controls that can withstand audit testing to demonstrate that the controls work. Creating and validating these controls is typically a time-consuming exercise. These controls exist within a framework of risks, which lead to the establishment of controls, which define activities and produce evidence, which can be used as feedback, to demonstrate management of the risk and improvement of the process.

An example of this framework is given in the next section.

After the framework is in place, the production of the annual Sarbanes-Oxley 404 report requires that data be produced for a set of defined controls for a business. The scenarios that appear later in this article outline the activities around a set of example controls for software-application development—showing how Visual Studio Team System 2008 contributes towards the production and reporting of evidence for these controls. For each of these scenarios, data would be produced in support of the production of the annual report for Sarbanes-Oxley 404 certification.

Outline of a Regulatory Framework in Action

While there is commonality among many businesses—such as order processing, accounts payable, manufacturing, R&D, distribution, sales and marketing, and so on—it is nonetheless necessary for a business to carry out an individual risk assessment for its own enterprise. The purpose for the business is to establish a set of specific risks that are relevant to Sarbanes-Oxley 404, and then to establish procedures that manage these risks successfully over time. Figure 1 shows the importance of the initial risk assessment and how the management of each specific risk relates to this:

Cc441754.c2c4ebd3-efbb-4c00-96ae-4ae2988f0f6b(en-us,VS.90).png

Figure 1. Regulatory framework for this business

The initial risk assessment is made by the business, in conjunction with its compliance expert. The scope of this assessment is broad—potentially, touching many areas across the firm. This might be a phased exercise, with the more significant business areas and risks being established first.

For each applicable risk that is identified by the business, a regulatory control objective is written that declares the intention of the risk management. For example, the specific risk might be that "the customer/business owner is not charged for changes that are made to the software," and the corresponding control objective would be "to guarantee that all chargeable changes to software are delivered for billing." A control activity is then put in place; this is a procedure with steps that fulfill this control objective—for example, to prepare a report regularly that indicates the billable status (yes or no) of every change to the system. The control activity also stipulates that the report be forwarded to the Accounting department for harmonization with its billing records. The reports and the records of their transmission to Accounting form the evidence for this control.

Using Visual Studio Team System 2008 to Support Compliance

Because regulatory frameworks span a wide range of business activities beyond the IT domain, Visual Studio Team System 2008 should be expected only to assist partially in the implementation of a regulatory framework within an organization. Visual Studio Team System 2008 is not a general-purpose compliance package; however, the following sections will demonstrate that it can aid in the production and collation of evidence that is required at the audit and testing stages of a compliance review or audit.

Figure 2 indicates the extent to which Visual Studio Team System 2008 can support the total Sarbanes-Oxley 404 compliance requirements of an organization:

Cc441754.266a8717-c092-44ce-90b8-cc61e765ee45(en-us,VS.90).png

Figure 2. Whole of business compliance (complete set of control objectives)

It is important to note that Sarbanes-Oxley 404 specifically relates to transactions that affect assets. In organizations in which software development has a significant financial impact, Visual Studio Team System 2008 can assist a business to capture and report evidence towards further demonstrating its compliance to Sarbanes-Oxley 404 requirements. While Sarbanes-Oxley 404 focuses on financial controls, other frameworks—such as COBIT, COSO, and ISO—have more general objectives, such as quality or industry-standards compliance. Visual Studio Team System 2008 also can assist in the capture and reporting of evidence in support of these other frameworks—using similar customizations, as outlined in the remainder of this article.

In addition, while Team System might be helpful in gathering information about the software-development process and producing a cumulative record of the development process, Visual Studio Team System 2008 does not embed rights-management functionality into such a cumulative record. Accordingly, in consultation with its Sarbanes-Oxley 404 expert, a business might want to consider enhancing the reliability of the records that are maintained/generated by Visual Studio Team System 2008 by, for example, including procedural controls (such as formal approval and recording) and education to project administrators and team members—advising them of the significance of the use of certain tools that might change or delete data, and what impact this might have on the overall compliance regime, and so on.

Introducing the Scenarios

This article presents a set of brief scenarios that indicate how Visual Studio Team System 2008 might be used to support some example controls. A reasonable degree of familiarity with the CMMI template for Visual Studio Team System 2008, and work items in general, is assumed. For our purposes, we will assume that the business under question has worked with a Sarbanes-Oxley 404 expert appraiser to develop a comprehensive risk analysis of its business.

As is typical for such an assessment, the resulting register might contain between 50 and 300 risks across the enterprise, for which controls will need to be implemented. Within this list are six risks that relate to software-application development in the IT department of this firm. These possible risk scenarios are as follows:

Table 1. Sarbanes-Oxley 404: Risk scenarios for application development

1.	Significant project risks are not identified or mitigated when they arise during the project.
2.	Unauthorized changes are incorporated into the project.
3.	The status of customer change requests is not tracked.
4.	There is a failure to understand or define customer requirements.
5.	The customer/business owner is not charged for changes to the system.
6.	Changes are not incorporated into the released system.

The following sections will examine the IT-related risk scenarios that are listed and will discuss their implementation, including Visual Studio Team System 2008. Extending these scenarios to cover wider Visual Studio Team System 2008 features (for example, source-code branching to manage releases) is outside of the scope of this article.

Scenario 1: Effective Management of Significant Project Risks

This scenario examines the specific area of the adequacy of risk management on software projects in the organization. If risk management on a software-development project is inadequate in practice, significant unchecked risks could lead to cost implications for the business and direct financial impacts on external stakeholders.

Item	Detail
Identified risk	Significant project risks are not identified and mitigated, as and when they arise during the life of the project.
Area	Risk management.
Control objective	Ensure that all significant project risks are identified and mitigated during the life cycle of the project.
Control activity	A risk register is maintained within Visual Studio Team System 2008 for the life of the project. The project manager ensures that risks are entered into Visual Studio Team System 2008, with any escalated risks being reviewed at the Project Risk Board.
"Evidence"	Risk records, Escalation reports and Risk-tracking records, minutes of the Project Risk Board.

Implementation Using Visual Studio Team System 2008

This scenario can be implemented in Visual Studio Team System 2008 by defining a process for the management of significant risks and adding a small number of additional fields to the CMMI Risk work item (see Fields Used, later in this section). In this context, we will define "significant" as any Risk work item that has a Severity of Critical or High. ("Severity" is an existing field.)

Outline of Required Process

The standard CMMI template for Visual Studio Team System 2008 includes a Risk work item. By adding the fields that are described later in this section, we are able to use the CMMI template to support the concept of a "significant risk register" for compliance purposes.

When a significant risk that is not easily closed arises in a project, it would be marked as "Escalated," and comments would be added to the History field. This allows risks to be reviewed by the Project Risk Review Board (or similar)—being a formally defined group that will review each significant risk for probable impact, and manage a plan for mitigation and contingency. The notes that are taken at these review meetings will be recorded in the (existing) History field in each Risk work item.

While the Iteration Path (existing field) could be used to infer in which iteration the risk is to be resolved, the new Target Date field is a more explicit means to declare the expected date by which this risk will have been mitigated to a non-significant state. It is expected that the Target Date field would be set by the Accountable Risk Owner (new field).

The (existing) Closed By and Closed Date fields should be set by the Accountable Risk Owner. Closure of the Risk is not considered the same as "Resolved by"/"Resolved date," which is the state before final approval is given to mark the Risk as closed. Closure would be approved by the Project Risk Board during a regular review meeting for significant risks. You can alter the work-item state workflow so that the Resolved to Closed transition can be performed only by the Accountable Risk Owner—being a member of a new Accountable Risk Owners security group. (Given that risk owners might include nontechnical personnel, the Web access client might be appropriate to provide access to Visual Studio Team System 2008.)

The business will be able to create a Project Risk Charter document, which defines the risk-management approach to significant risks. This would include a list of Project Risk Board members, which would match the list of members within the Accountable Risk Owners security group. Any changes to the Project Risk Board membership would be reflected in this document, and the security group would be updated on the same day. The Project Risk Charter would indicate also when the Project Risk Board would meet. The Project Risk Charter document also would define "significant" as being the High or Critical value in the Risk Severity field in the CMMI template. The Project Risk Charter also would include details of where the Visual Studio Team System 2008 server was located, which Visual Studio Team System 2008 project names were overseen by the Board, and so on.

Notes or minutes that result from the Project Risk Board meetings could be a useful component of any compliance framework. These minutes may be attached to the Risk work item (using the existing Attachments field), or alternatively a resource pointer (URL or file reference) may be entered into the History field (such as a pointer to a document within Microsoft Office SharePoint). The minutes might document, for example, whether mitigation and contingency plans in the Risk work item have been reviewed and updated, or other ways in which risk is being managed actively.

The business will be able to use Team System to help track that significant project risks are being managed in a structured and accountable way by the use of queries, as in the following list:

Produce a list of the risks with Severity of High and Critical.
Examine each Risk by its State field, so that Active, Resolved, and Closed risks might all be examined.
By using the Links field, link from each Risk in this list to the corresponding Task work items, to demonstrate project activity that takes place over time to address each risk. A query could be implemented in Office Excel to show the linked Tasks that derive from a parent Risk.
Examine the History field of each Risk work item to reveal the actual changes to the Risk over time—satisfying the business that real activity was taking place during the life of the Risk; for example, we might examine what reasons were recorded in the History at the time that the Risk was marked as "Escalated."
Compare the Resolved and Closed dates for Risks, to ensure that a Project Risk Board meeting took place between these (that is, not Resolved and Closed on the same day).
Compare the Closed and Target dates, to examine the efficacy of risk-management timing.
Compare the Closed By field to the formally documented Accountable Risk Owners security group.

The preceding list is not an exhaustive one, but it is a high-level example of what can be achieved with Visual Studio Team System 2008. It might be taken as a starting point for implementing a formal project risk register for software-application development projects.

You will see from this scenario that a combination of human activity and tool support is needed to support the aims of the regulatory framework for a business. The benefit of using Visual Studio Team System 2008 is in the proximity of the tool to the real daily activity on the project; the information that is captured is more likely to be reflective of what actually takes place, and tends to encourage better enactment of team processes (such as risk management) due to this close shadowing of team software development.

Fields Used

In addition to the existing fields in the Risk work item, the following fields may be added by the customization of the CMMI template:

New field	Notes
Accountable Risk Owner	Field that represents the name of the person who is responsible for the life of the risk and its end result; this name must be a member of a new Accountable Risk Owners security group.
Escalated	Flag field that indicates whether this risk has been escalated to the Project Risk Board for review. All risks for which this field is set affirmatively will be reviewed at a regular Project Risk Board meeting.
Target Date	Field that identifies specifically a target date by which the risk should have been closed.

Example

This section illustrates some views of the Visual Studio Team System 2008 client, and shows a path to implement the Risk methodology that was described earlier. These views are possible after the customizations to the Risk work item have been made in the CMMI template.

Figure 3 shows a query that is performed to examine project Risks. The three additional fields that are added to the Risk work item—Accountable Risk Owner, Escalated, and Target Date—appear highlighted in yellow:

Cc441754.a4a38186-2da7-483d-af30-bac817ea4f61(en-us,VS.90).gif

Figure 3. Query examining project risks

This query produces a list of eight Risks that are classified in Severity as either Critical or High. The first item in the list is highlighted, and the work-item form is shown below the list. Each Risk has an Accountable Risk Owner who takes formal responsibility for seeing it through to completion, along with a Target date for resolution as described in the preceding sections (see Outline of Required Process).

The query can provide any set of relevant fields, with specific filters to limit the records that are displayed. Queries may be built readily in Visual Studio Team Explorer; this delivers a benefit in being able to create queries rapidly for auditing purposes.

Figure 4 shows the definition of a query that is used to show the "SOX 404 Escalated Risks":

Cc441754.544aaec0-cc48-4772-80da-de753863fd71(en-us,VS.90).gif

Figure 4. Definition of query used for showing "SOX 404 Escalated Risks"

The preceding image shows the criteria for selecting the aforementioned Risks. In this case, the user has elected to view only Risk work items that are either Critical or High in Severity, and that have a State of Active, Resolved, or Closed. In addition, the user is interested only in Risks that have been marked as Escalated to the Project Risk Board, so the user includes this in the query definition.

The fields that are shown in the results list may be configured separately from the query definition itself. This allows additional data to be shown for sorting purposes.

Figure 5 shows the dialog box for choosing the fields for the query result set:

Cc441754.dd5a6d65-7176-4227-a472-a753f8c19a51(en-us,VS.90).gif

Figure 5. Column Options dialog box, for choosing fields for query result set

Scenario 2: Prevention of Unauthorized Changes to the Project

This scenario explores the level of rigor that is associated with the process of making changes to the software that is under development. When changes are not managed closely, discrepancies can arise over delivered functionality when compared to that which was contracted. When changes cannot be accounted for clearly, these can have serious implications—for example, in project scheduling, security, and destabilization or interference with existing functionality.

Item	Detail
Identified risk	Unauthorized changes are incorporated into the project.
Area	Change management.
Control objective	Ensure that all requests for project changes are authorized through formal procedures and that they do not compromise the project.
Control activity	Change Requests are approved according to the Project Change Board terms of reference (a written document). All Change Requests require that an impact and risk assessment be performed. The Change Board is responsible for ensuring that the appropriate impact and risk analyses have been completed as part of their change review and approval process. Relevant information is included in every Change Request document.
"Evidence"	Approved Change Requests, and copies of Project Change Board minutes that show review of Change Requests.

Implementation Using Visual Studio Team System 2008

The existing CMMI template already includes a Change Request work item. With the addition of some extra fields (see Fields Used, later in this section) and changes to current fields, it is possible to offer a change-request system with an increased level of provability.