Agile SDL

Streamline Security Practices For Agile Development

Bryan Sullivan

Contents

A New Process: Combining the SDL and Agile
The Every-Sprint Requirements
The Onboarding Requirements
The Bucket Requirements
Constraints
Wrap-Up

In the September 2008 issue of MSDN Magazine, I wrote a column about the additions that Microsoft has made to the Security Development Lifecycle (SDL) process to address security vulnerabilities in online services. I talked about the importance of input validation and output encoding in order to prevent cross-site scripting attacks; about using parameterized stored procedures and restricting database permissions in order to defend against SQL injection attacks. I also discussed restricting the use of wildcards in cross-domain policy files so you can defend against request forgery attacks (see "SDL Embraces The Web").

All of these SDL additions are necessary to protect your Web apps, but they raise challenges for Web app development teams. In order to make the SDL more practical for Web app and online services teams, the process itself needs to change to better fit the development processes that those teams use. In other words, it's not just that the SDL needs new, Web-specific requirements; it's that those requirements need to be applied in a different manner as well.

The biggest difficulty in adapting the SDL to the needs of Web applications is simply one of time. The SDL was originally developed to improve the security of large, complex products like Windows, Microsoft Office, and SQL Server, and it has done so very successfully. Part of the reason for its success is its thoroughness: in its latest version, the SDL has more than 80 separate requirements and recommendations that product teams follow to improve their products' security and privacy. Figure 1 gives you an idea of the process involved in the SDL.

Figure 1 The Classic Security Development Lifecycle

For a product like SQL Server that has a two- to three-year development cycle, completing 80 requirements over the course of the release schedule is not an unfair burden. However, most Web applications don't have such long development cycles; they have two- to three-week development cycles. (In fact, some online services teams at Microsoft have development cycles as short as one week.) So while it may not be a problem for Office to complete 80 SDL requirements over three years, it would be completely unreasonable to expect an online service like Office Live to complete 80 requirements every three weeks. And even if they could complete this much security work, they wouldn't have much time left to implement any new features. Security is vitally important, but we can't focus on security to the complete exclusion of everything else.

A New Process: Combining the SDL and Agile

To address this issue, the SDL team has adapted the SDL to better suit the needs of teams with Rapid Application Development (RAD) or Agile development processes. We haven't yet determined an official name for this new SDL variant, but for the purposes of this article we'll refer to it as SDL/Agile, or SDL/A for short. Our intention is that any product team with an iterative, phaseless development methodology will be eligible to opt into the SDL/A process instead of the "classic" version of SDL. This does not limit SDL/A to only Web applications, since many box products also follow Agile development methodologies.

The first and biggest difference between classic SDL and SDL/A is that in SDL/A, every requirement does not need to be completed for every release (or every "sprint"). This may seem controversial. After all, we've always said that every requirement in the SDL is important and can't be skipped.

It is true that every SDL requirement is included because it has been proven to prevent a serious security or privacy vulnerability. The requirement to avoid certain banned APIs is included because it has been proven to prevent buffer overflows. The requirement to encode user input before displaying it on a Web page has been proven to prevent cross-site scripting attacks. However, as I discussed, there simply isn't enough time in a short release cycle to complete every SDL requirement and still do feature development. So, SDL/A defines three levels of requirement frequency (that is, how often the requirement must be completed), and each SDL requirement is placed into one of those three categories.

The Every-Sprint Requirements

The first frequency level for SDL/A is the "every-sprint" level. These requirements are considered non-negotiable and must be completed every sprint regardless of how short the sprint length is. These requirements place the highest burden on the development teams since they're being completed so frequently, so these requirements are chosen very carefully according to two factors.

The first factor is the importance of the requirement—in other words, how many vulnerabilities does this requirement prevent, and how critical are those vulnerabilities? Every requirement is important, but just as different vulnerabilities have different weights on the bug bar, the requirements that defend against those vulnerabilities have different weights as well.

For example, a temporary Denial of Service (DoS) attack is considered a much less severe attack than a remotely executed elevation of privilege. An SDL requirement that would prevent the elevation of privilege attack is much more likely to be included in the every-sprint frequency category. We also look at historic vulnerability data: if a requirement would have prevented one of the critical security vulnerabilities, such as those that were exploited by the Slammer or Blaster worms (if the requirement had existed in time to prevent introduction of the vulnerability), then that requirement is almost certain to be included in the every-sprint category.

One example of a requirement that falls into the every-sprint category is the requirement to use the ASP.NET-provided Validate­Request cross-site scripting (XSS) defense. The ValidateRequest requirement meets both criteria for inclusion in every sprint: it helps products to defend against highly critical XSS attacks, and it's essentially free since ValidateRequest is enabled by default.

The second factor that contributes to a requirement being placed in the every-sprint category is the ease with which that requirement can be automated. Even if a requirement is unlikely to prevent another Code Red attack, if it can be automated into the development build process or code check-in policy, then it's likely to be required every sprint.

Another example of an every-sprint requirement is the requirement to threat model the application. Unlike enabling Validate­Request, threat modeling can take a significant amount of time and effort. However, threat modeling is the cornerstone of the SDL and can never be omitted. Threat modeling is essential to uncovering design-level vulnerabilities that might require significant re-architecture in order to be mitigated if they're found later in the development lifecycle. Like heart surgery, threat modeling is expensive, but it's well worth the price!

In many cases, the cost of threat modeling is not even that high. One of the tenets of the Agile development process is to plan for no more work than is going to be completed during the current sprint. This goes for threat modeling too. There's no need to create a detailed threat model of the complete system during the very first sprint. Only the functionality being developed in the current sprint needs to be threat modeled for that sprint. In this way, the threat modeling requirement is self-regulating with regard to time spent.

The Onboarding Requirements

The second group of requirements are referred to as "onboarding" requirements. These are requirements that product teams have to complete once at the beginning of the project and then never need to address again. In the classic SDL, every requirement works this way. But products using classic SDL typically have well-defined development phases, including a well-defined end (release of the product). In contrast, Web applications often do not have well-defined endings. The team may release a new update every Friday afternoon and then start developing new features Monday morning, and this process can continue indefinitely.

An example of a one-time-only onboarding requirement is the requirement to configure the product's bug tracking system to track security-specific data. At Microsoft, all security defects found during development that are rated as Critical, Important, or Moderate in the product's bug bar must be fixed before release. But if you can't track which defects are security issues and which are functional issues, you can't reliably fix the security issues before release. Additionally, the security-specific defect fields, such as Security Defect Cause (buffer overflow, directory traversal, SQL injection, and so on) and Security Defect Effect (any of the STRIDE values), can be valuable to help developers fix the defect and to ensure that it doesn't happen again later. The team only needs to set up the tracking system this way once, no matter how long the project runs.

Another, slightly more involved onboarding requirement is the requirement to create a baseline threat model for the application. If the application is a new project, if no code has ever been written for it before, then this requirement is superfluous: the team only needs to threat model new features as they're designed on a sprint-by-sprint basis. However, a product team may be applying the SDL/A process for the first time to a previously existing project. In this case, a baseline threat model must be built to cover the preexisting functionality of the application.

Several other onboarding requirements, such as developing a security incident response plan, can be nontrivial as well. In order to prevent teams from spending a disproportionate amount of time on security, especially in their first stages with a new, unfamiliar SDL process, SDL/A provides for a three-month grace period for teams to complete the onboarding requirements.

The Bucket Requirements

All other SDL requirements that do not fall into either the every-sprint or onboarding requirement categories are placed into one of three requirement buckets: Security Verification, Design Review, and Response Plans. So, for example, the SDL requirement to fuzz input handling routines would be placed in the Security Verification bucket. Likewise, the requirement to define a disaster recovery plan would be placed in the Response Plans bucket.

Unlike in classic SDL, where all of these requirements must be completed before the product can release, in SDL/A only one requirement from each bucket must be completed during each sprint. This is the concession that SDL/A makes to the shorter release schedules of Agile development projects. As they say, you can't fit ten pounds of flour in a five-pound sack, and the bucket requirements are the five extra pounds we've taken out. Figure 2 shows the new flow with SDL/A. Note that the checked items will be performed during the corresponding stage.

Figure 2 SDL/A Lifecycle

However, even though product teams aren't required to complete every bucket requirement in every sprint, that doesn't mean they can omit them indefinitely. In fact, teams have to complete each of the bucket requirements at least once a year. Additionally, teams are prohibited from completing the same bucket requirement two sprints in a row. For example, if Project X's team chooses to complete an attack surface analysis to satisfy its Security Verification bucket requirement for their March sprint, they can't then complete another attack surface analysis in April. More specifically, they could complete another attack surface analysis in April if they really wanted to; nevertheless, it just would not satisfy their SDL commitment.

Other than those two constraints, teams are free to choose whichever bucket requirements they want to complete in any given sprint. SDL/A does not mandate any sort of ordering or round-robin requirement selection. This gives product teams maximum flexibility in choosing security activities that they find most useful. So if a team has historically found a significant number of defects by running Application Verifier but they've never found any by running a SOAP fuzzer, they can choose to run AppVerif as often as every other sprint but need only run the SOAP fuzzer once a year.

Constraints

While it's not officially prohibited, teams that follow a waterfall-style development process with well-defined and distinct design, implementation, verification and release phases are not a good fit for SDL/A because it is designed to work with iterative, phaseless development methodologies. If a product has well-defined phases, then classic SDL is a much better fit.

Wrap-Up

The concept of Agile security does not have to be a contradiction in terms. With SDL/A, the Microsoft SDL team has defined a set of process improvements that provide for an increased security focus while still respecting the need to release new code (including new functionality, not just new security improvements) on an ultra-short timeline. If you have experience, whether positive or negative, completing the classic SDL requirements in a short sprint, I'd be more than happy to hear your thoughts on SDL/Agile; you can reach me at blogs.msdn.com/sdl. Also, please see the official SDL site at www.microsoft.com/sdl.

Bryan Sullivan is a Security Program Manager for the Microsoft Security Development Lifecycle team, where he specializes in Web application security issues. His first book, Ajax Security, was published by Addison-Wesley in December 2007.