Security Briefs

Reinvigorate your Threat Modeling Process

Adam Shostack

Contents

Approaches to Threat Modeling
A Quick and Dirty Threat Model
Knowing When You're Finished
Driving the Threat Modeling Process
Keeping It in Sync: Dealing with Change
Help, I'm Lost!

My colleague Ellen likes to say that everyone threat models all the time. We all threat model airport security. We all threat model our homes. We think about threats against our assets: our families, our jewelry, and our sentimental and irreplaceable photographs (well, those of us old enough to have photos that never existed in digital form do). We model threats based on architecture: there's a wall here, a picture window there, and an easily climbed tree that we can use when we forget our keys. And we model threats based on attackers. We worry about burglars and kids falling into pools. We also worry about the weather, be it earthquakes, snow, or tornadoes.

If I wanted to sound like a management consultant, I'd say you employ a mature, multi-dimensional assessment process, with a heavy reliance on heuristics and low reproducibility across instances. At the same time, it's likely you won't have thought of everything or implemented defenses against every possible attack. It's very unlikely you have a home defense management plan or have ever run a penetration test against your home.

As we build software, regardless of whether we're in an agile or a waterfall world, we need agreement on what we're building, what we're not building, and what we're doing to ensure we're building the right thing. In the past few years, a perception that threat modeling is a heavy, bureaucratic process has been generated. There are some good reasons to move toward adding processes; I'd like to talk about them, some lessons learned from these processes, and how to put the fun back in threat modeling while making it an efficient, agile-friendly activity that anyone can do.

Approaches to Threat Modeling

There are many things called threat modeling. Rather than argue about which is "the one true way," consider your needs and what your skills, abilities, and schedules are, and then work with a method that's best for you. As part of that approach, some people ask, "What's your threat model?" and "Have you threat modeled that component?" One is requirements elicitation, the other design analysis. At Microsoft, we almost always mean the latter technique.

There are more threat modeling methods out there than I can dream of covering in one column. There's also a tremendous diversity of goals. Should your threat modeling process be fast or deep? Should it focus on assurance and completeness, or ease of use? Should you involve experts or developers in every meeting? Do you have organizational or industry rules you need to follow, such as the Microsoft® Security Development Lifecycle (SDL) or the rules for medical device manufacturers? The high level objective should be to understand security issues early so you can address them in the design rather than try to overcome design flaws later.

Some of the major ways to approach threat modeling activity include the following:

Assets Asset-driven threat modeling is much like thinking about what you want to protect in your house. You start by listing what assets your software has associated with it, and then you think about how an attacker might compromise those assets. Examples include a database that stores customer credit cards or a file that contains encrypted passwords.

Some people may interpret an asset as an element of the threat modeling diagram, thinking that a Web server itself is an asset. That way lies madness. Digital assets are things an attacker wants to read, tamper with, or deny you the use of.

Attackers Attacker-driven threat modeling involves thinking about who might want your assets, and it works from an understanding of their capabilities to an understanding of how they might attack you. This works great when your adversary is a foreign army with a known strategic doctrine, physical world limits, and long-lead-time weapons systems development. This works less well when your adversary is a loosely organized group of anonymous hackers.

More generally, it's not clear this is useful in software threat modeling. There are certainly people for whom "think like an attacker" is an effective part of design analysis. It's less clear that this is a reproducible process in which people can get training. If you're going to start from attackers, it's probably worth using a standard set. It will be helpful to have a small set of these anti-personas written out.

Software Design Design-driven threat modeling is threat modeling based on where your fences and windows are. You draw diagrams and worry about what can go wrong with each thing in your diagram. (This is the essence of the SDL threat modeling process today because everyone in software knows how to draw diagrams on a whiteboard.)

The software equivalents of fences and windows are the various forms of attack surface, such as file parsers or network listening services—sockets, remote procedure call (RPC) services, Web services description language (WSDL) interfaces, or AJAX APIs. They're the trust boundaries where you should expect an attacker to first get a foothold.

A Quick and Dirty Threat Model

Threat modeling doesn't have to be a chore. Following the process illustrated in Figure 1, here is the outline of a basic threat modeling process that will get you going quickly and painlessly:

  1. Diagram your application, and use this to tell your app's story in front of the whiteboard (see Figure 2). Use circles for code, boxes for things that exist outside of it (people, servers), and drums for storage. Our team uses funny looking parallel lines for data stores. Draw some trust boundaries using dotted lines to distinguish domains.
  2. Identify threats by brainstorming. When you get stuck, apply the STRIDE threat model, described in Figure 3, on each element of your app. Don't worry about the fixes, just get a brainstorming flow going.
  3. Consider redesigns by considering whether the threats are congregating or are totally scattered. Either way, you might want to design a component out or add one to address them centrally. All the threats in one place may mean you're worried about the front door and not worrying about anything else.
  4. Mitigate the first order threats. Go broad, not deep. It can be helpful to think of threats and mitigations as ordered. A first order threat is opening the door. The first order mitigation is a lock. The second order threat is that someone picks the lock or breaks it down. Second order mitigations include good locks and solid doors, properly installed. A third order defense might be an alarm system on the door, and to mitigate the threat of someone cutting the wire, you send a regular message down the wire. If you find yourself worrying about the software equivalent of what happens when someone cuts the phone wire to the alarm system before you worry about locks on the doors, you're worrying about the wrong things.
  5. File bugs so you can fix what you found threat modeling.

fig01.gif

Figure 1 The Threat Modeling Process (Click the image for a larger view)

fig02.gif

Figure 2 Diagramming Your App Helps Threat Modeling (Click the image for a larger view)

Figure 3 STRIDE Threat Model

Threat Property Definition Example
Spoofing Authentication Impersonating something or someone else. Pretending to be any of these: billg, microsoft.com, or ntdll.dll.
Tampering Integrity Modifying data or code. Modifying a DLL on disk or DVD, or a packet as it traverses the LAN.
Repudiation Non-repudiation Claiming to have not performed an action. "I didn't send that e-mail," "I didn't modify that file," "I certainly didn't visit that Web site, dear!"
Information Disclosure Confidentiality Exposing information to someone not authorized to see it. Allowing someone to read the Windows source code; publishing a list of customers to a Web site.
Denial of Service Availability Deny or degrade service to users. Crashing Windows or a Web site, sending a packet and absorbing seconds of CPU time, or routing packets into a black hole.
Elevation of Privilege Authorization Gain capabilities without proper authorization. Allowing a remote Internet user to run commands is the classic example, but going from a limited user to admin is also EoP.

Knowing When You're Finished

When your diagrams show all of the relevant parts of the system, and you've walked through the STRIDE model for every element in the diagrams, you're doing pretty well. (You're doing even better if you've filed bugs for each vulnerability.)

However, that's somewhat like saying you're finished when you've checked that all the doors and all the windows have locks. That may be enough. It may be what you have time to do. Or maybe you're planning to go deeper, to think about bypasses for the defenses you've put in place.

How deep you go depends a lot on your organization's risk tolerance, or that of your customers. It also depends on your budget and experience. It may be that you want to enter into a threat modeling process, prove success, and exit reasonably quickly. It may be that you're designing the control system for something safety critical and need to dive deep and spend a good deal of time on it.

So knowing when you're done involves a set of trade-offs. First you need to consider the policies, laws, or regulations mandating your level of effort. Then you need to assess the risk for the system being developed. Finally, you need to account for the availability of time and resources both for your threat modeling process and any resulting mitigation and testing.

Driving the Threat Modeling Process

Threat modeling doesn't just happen by itself. Someone needs to drive it. Who's driving it will have a major impact on how it works. If you have one or even a few people who love security, they can drive threat modeling across an organization.

There's a question of how or whether to involve other people. This is both a cultural and a resources question. If the enthusiasts can threat model everything, share the results, and get the issues detected fixed, then great! Why involve others?

Unfortunately, sometimes there are more projects than security enthusiasts to take them on, and you need to bring others in. Microsoft has found that threat modeling works better with a security expert in the room, but there isn't always one available.

Your expert is someone who thinks about security all the time and can do so in a structured way. It's someone who can talk about what a repudiation threat might mean in the context of your design. Your expert might have a certification of some sort. He might be that someone who points out security flaws in your code and operations. She might attend (or want to attend) conferences such as SANS or BlackHat, or she may have been thanked in a Microsoft Security Bulletin.

You can get decent results by giving people structure and feedback on their work, and by breaking it down into small, easy pieces with rules and self-checks in each one. However, breaking threat modeling down into too many little pieces and having too many rules (or the wrong rules) can take all the fun out of the process, and that turns threat modeling into a chore. So how you break the pieces out depends on how many experts you have and how much time they can dedicate to the process.

Keeping It in Sync: Dealing with Change

Plans change. It's unavoidable. When they do, there's a cost to keeping the documentation synchronized with the many real plans that are in everyone's heads. No matter how agile you are, a project of any substantial size is definitely going to have some architecture documentation somewhere. Maybe it's a set of Visio® diagrams the size of a large wall, or maybe it turns out to be a whiteboard that can't be erased.

For threat modeling purposes, you want to know when things you do add trust boundaries to that architecture. When you do, you have to consider the implications of those new threats.

Sometimes, the threats are direct. Say you change a component to listen on port 80. When threats are direct to the component, you can easily discover them, address them, and move on.

Other times, the threats are indirect. Say you decide to get configuration data from Active Directory® and Lightweight Directory Access Protocol (LDAP). This probably entails a few new components (the directory interface and an abstraction layer so you can switch), but it also entails a change to the configuration data. Before, the data might have been entered by an administrative user into your GUI. Now it's coming in over the network, using several different protocols. It's probable that you'll want to validate the data very carefully in the abstraction layer, and perhaps also become more cautious about how you read and act on configuration data.

Help, I'm Lost!

You may find yourself sitting around a table saying, "What do we do now?" Maybe you're having trouble understanding the process. Maybe one step has you completely stumped. Here are some problems we've seen on the SDL team and how to get unstuck:

Some teams have problems with diagramming. If you don't know what the system does, find someone who does, or break the threat modeling process and spend time understanding this thing you've inherited.

If you don't know what the system should do—perhaps you're working on a new system, and you have two competing designs—threat modeling can help unstick your decision-making progress. Draw both designs and ask, "Which has the worst vulnerabilities?"

If you don't know how to represent an element, don't sweat it too much. You may use the wrong element type, and you'll notice. But in essence, all the code that you are responsible for, no matter who wrote it, is a process. All data that you read and write to is stored in data stores.

You may also come across problems enumerating threats. First, it's important you don't let yourself get stuck on a single threat. Flag it and come back to it later. If that doesn't work, check out Larry Osterman's great series on threat modeling (go.microsoft.com/fwlink/?LinkId=118281). There's also good stuff in Michael Howard's books Writing Secure Code and The Security Development Lifecycle.

For problems validating the threat model and your mitigation plan, look to see whether the diagrams represent the code and whether you have agreement between developers and testers on that. Look to see if you've enumerated a threat for each threat type against each element (see the November 2006 MSDN® Magazine article "Uncover Security Design Flaws Using the STRIDE Approach," available at msdn.microsoft.com/magazine/cc163519). And finally, check to see if you've filed and fixed a bug per threat.

If you happen to have an expert handy, ask him for help. Even our experts ask each other, "How would you approach this issue?" If you don't have an expert handy, try getting one, either by hiring him or blogging about your problem.

Don't let yourself get stymied by a single issue. Threat modeling is too important for you to skip. And remember to have fun. Breaking stuff before you ship is not only fun, it's a heck of a lot more fun than fixing it afterward.

Send your questions and comments to briefs@microsoft.com.

Adam Shostack is a Program Manager on the Security Development Life­cycle (SDL) team at Microsoft. He is responsible for the threat modeling component of the SDL.