December 2010

Volume 25 Number 12

BDD Primer - Behavior-Driven Development with SpecFlow and WatiN

By Brandon Satrom | December 2010

As automated unit testing becomes more ubiquitous in software development, so does the adoption of various test-first methods. These practices each present a unique set of opportunities and challenges to development teams, but all strive to establish a “testing as design” mindset with practitioners.

For much of the test-first era, however, the method for expressing the behavior of a user has been through unit tests written in the language of the system—a language disconnected from that of the user. With the advent of Behavior-Driven Development (BDD) techniques, this dynamic is changing. Using BDD techniques you can author automated tests in the language of the business, all while maintaining a connection to your implemented system.

Of course, a number of tools have been created to help you implement BDD in your development process. These include Cucumber in Ruby and SpecFlow and WatiN for the Microsoft .NET Framework. SpecFlow helps you write and execute specifications within Visual Studio, while WatiN enables you to drive the browser for automated end-to-end system testing.

In this article, I’ll provide a brief overview of BDD and then explain how the BDD cycle wraps the traditional Test-Driven Development (TDD) cycle with feature-level tests that drive unit-level implementation. Once I’ve laid the groundwork for test-first methods, I’ll introduce SpecFlow and WatiN and show you examples of how these tools can be used with MSTest to implement BDD for your projects.

A Brief History of Automated Testing

One of the most valuable practices to emerge from the Agile Software movement is an automated, test-first development style, often referred to as Test-Driven Development, or TDD. A key tenet of TDD is that test creation is as much about design and development guidance as it is about verification and regression. It’s also about using the test to specify a unit of required functionality, and using that test to then write only the code needed to deliver that functionality. Therefore, the first step in implementing any new functionality is to describe your expectations with a failing test (see Figure 1).

Figure 1 The Test-Driven Development Cycle

Many developers and teams have had great success with TDD. Others have not, and find that they struggle with managing the process over time, especially as the volume of tests begins to grow and the flexibility of those tests begins to degrade. Some aren’t sure how to start with TDD, while others find TDD easy to initiate, only to watch it abandoned as deadlines near and large backlogs loom. Finally, many interested developers meet resistance to the practice within their organizations, either because the word “test” implies a function that belongs on another team or because of the false perception that TDD results in too much extra code and slows down projects.

Steve Freeman and Nat Pryce, in their book, “Growing Object-Oriented Software, Guided by Tests” (Addison-Wesley Professional, 2009), note that “traditional” TDD misses some of the benefits of true test-first development:

“It is tempting to start the TDD process by writing unit tests for classes in the application. This is better than having no tests at all and can catch those basic programming errors that we all know but find so hard to avoid … But a project with only unit tests is missing out on critical benefits of the TDD process. We’ve seen projects with high-quality, well unit-tested code that turned out not to be called from anywhere, or that could not be integrated with the rest of the system and had to be rewritten.”

In 2006, Dan North documented many of these challenges in an article in Better Software magazine (dannorth.net/introducing-bdd). In his article, North described a series of practices that he had adopted over the prior three years while in the trenches with testing. While still TDD by definition, these practices led North to adopt a more analysis-centric view of testing and to coin the term Behavior-Driven Development to encapsulate this shift.

One popular application of BDD attempts to extend TDD by tightening the focus and process of creating tests through Acceptance Tests, or executable specifications. Each specification serves as an entry point into the development cycle and describes, from the user’s point of view and in a step-by-step form, how the system should behave. Once written, the developer uses the specification and their existing TDD process to implement just enough production code to yield a passing scenario (see Figure 2).

Figure 2 The Behavior-Driven Development Cycle

Where Design Begins

BDD is considered by many a superset of TDD, not a replacement for it. The key difference is the focus on initial design and test creation. Rather than focusing on tests against units or objects, as with TDD, I focus on the goals of my users and the steps they take to achieve those goals. Because I’m no longer starting with tests of small units, I’m less inclined to speculate on fine-grained usage or design details. Rather, I’m documenting executable specifications that prove out my system. I still write unit tests, but BDD encourages an outside-in approach that starts with a full description of the feature to be implemented.

Let’s look at an example of the difference. In a traditional TDD practice, you could write the test in Figure 3 to exercise the Create method of a CustomersController.

Figure 3 Unit Test for Creating a Customer

[TestMethod]
public void PostCreateShouldSaveCustomerAndReturnDetailsView() {
  var customersController = new CustomersController();
  var customer = new Customer {
    Name = "Hugo Reyes",
    Email = "hreyes@dharmainitiative.com",
    Phone = "720-123-5477" 
  };
  var result = customersController.Create(customer) as ViewResult;
  Assert.IsNotNull(result);
  Assert.AreEqual("Details", result.ViewName);
  Assert.IsInstanceOfType(result.ViewData.Model, typeof(Customer));
  customer = result.ViewData.Model as Customer;
  Assert.IsNotNull(customer);
  Assert.IsTrue(customer.Id > 0);
}

With TDD, this tends to be one of the first tests I write. I design a public API to my CustomersController object by setting expectations of how it will behave. With BDD I still create that test, but not at first. Instead, I elevate the focus to feature-level functionality by writing something more like Figure 4. I then use that scenario as guidance toward implementing each unit of code needed to make this scenario pass.

Figure 4 Feature-Level Specification

Feature: Create a new customer
  In order to improve customer service and visibility
  As a site administrator
  I want to be able to create, view and manage customer records
Scenario: Create a basic customer record
  Given I am logged into the site as an administrator
  When I click the "Create New Customer" link
  And I enter the following information
    | Field | Value                       |
    | Name  | Hugo Reyes                  |
    | Email | hreyes@dharmainitiative.com |
    | Phone | 720-123-5477                |
  And I click the "Create" button
  Then I should see the following details on the screen:
    | Value                       |
    | Hugo Reyes                  |
    | hreyes@dharmainitiative.com |
    | 720-123-5477                |

This is the outer loop in Figure 2, the failing Acceptance Test. Once this test has been created and fails, I implement each step of each scenario in my feature by following the inner TDD loop depicted in Figure 2. In the case of the CustomersController in Figure 3, I’ll write this test once I reach the proper step in my feature, but before I implement the controller logic needed to make that step pass.

BDD and Automated Testing

From the start, the BDD community has sought to provide the same level of automated testing with Acceptance Tests that has been the norm in unit testing for some time. One notable example is Cucumber (cukes.info), a Ruby-based testing tool that emphasizes the creation of feature-level Acceptance Tests written in a “business-readable, domain-specific language.”

Cucumber tests are written using User Story syntax for each feature file and a Given, When, Then (GWT) syntax for each scenario. (For details on User Story syntax, see c2.com/cgi/wiki?UserStory.) GWT describes the current context of the scenario (Given), the actions taken as a part of the test (When) and the expected, observable results (Then). The feature in Figure 4 is an example of such syntax.

In Cucumber, user-readable feature files are parsed, and each scenario step is matched to Ruby code that exercises the public interfaces of the system in question and determines if that step passes or fails.

In recent years, innovations enabling the use of scenarios as automated tests have extended into the .NET Framework ecosystem. Developers now have tools that enable specifications to be written using the same structured English syntax that Cucumber utilizes, and which can then use those specifications as tests that exercise the code. BDD testing tools like SpecFlow (specflow.org), Cuke4Nuke (github.com/richardlawrence/Cuke4Nuke) and others enable you to create executable specifications first in the process, leverage those specifications as you build out functionality and end with a documented feature that’s tied directly to your development and testing process.

Getting Started with SpecFlow and WatiN

In this article, I’ll utilize SpecFlow to test a Model-View-Controller (MVC) application. To get started with SpecFlow, you’ll first need to download and install it. Once SpecFlow is installed, create a new ASP.NET MVC application with a unit test project. I prefer that my unit test project contain only unit tests (controller tests, repository tests and so on), so I also create an AcceptanceTests test project for my SpecFlow tests.

Once you’ve added an AcceptanceTests project and added references to the TechTalk.SpecFlow assembly, add a new Feature using the Add | New Item templates that SpecFlow creates on installation and name it CreateCustomer.feature.

Notice that the file is created with a .feature extension, and that Visual Studio recognizes this as a supported file, thanks to SpecFlow’s integrated tooling. You may also notice that your feature file has a related .cs code-behind file. Each time you save a .feature file, SpecFlow parses the file and converts the text in that file into a test fixture. The code in the associated .cs file represents that test fixture, which is the code that’s actually executed each time you run your test suite.

By default, SpecFlow uses NUnit as its test-runner, but it also supports MSTest with a simple configuration change. All you need to do is add an app.config file to your test project and add the following elements:

<configSections>
  <section name="specFlow"
    type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow"/>
</configSections>
<specFlow>
  <unitTestProvider name="MsTest" />
</specFlow>

Your First Acceptance Test

When you create a new feature, SpecFlow populates that file with default text to illustrate the syntax used to describe a feature. Replace the default text in your CreateCustomer.feature file with the text in Figure 4.

Each feature file has two parts. The first part is the feature name and description at the top, which uses User Story syntax to describe the role of the user, the user’s goal, and the types of things the user needs to be able to do to achieve that goal in the system. This section is required by SpecFlow to auto-generate tests, but the content itself is not used in those tests.

The second part of each feature file is one or more scenarios. Each scenario is used to generate a test method in the associated .feature.cs file, as shown in Figure 5, and each step within a scenario is passed to the SpecFlow test runner, which performs a RegEx-based match of the step to an entry in a SpecFlow file called a Step Definition file.

Figure 5 Test Method Generated by SpecFlow

public virtual void CreateABasicCustomerRecord() {
  TechTalk.SpecFlow.ScenarioInfo scenarioInfo = 
    new TechTalk.SpecFlow.ScenarioInfo(
    "Create a basic customer record", ((string[])(null)));
  this.ScenarioSetup(scenarioInfo);
  testRunner.Given(
    "I am logged into the site as an administrator");
  testRunner.When("I click the \"Create New Customer\" link");
  TechTalk.SpecFlow.Table table1 = 
    new TechTalk.SpecFlow.Table(new string[] {
    "Field", "Value"});
  table1.AddRow(new string[] {
    "Name", "Hugo Reyesv"});
  table1.AddRow(new string[] {
    "Email", "hreyes@dharmainitiative.com"});
  table1.AddRow(new string[] {
    "Phone", "720-123-5477"});
  testRunner.And("I enter the following information", 
    ((string)(null)), table1);
  testRunner.And("I click the \"Create\" button");
  TechTalk.SpecFlow.Table table2 = 
   new TechTalk.SpecFlow.Table(new string[] {
  "Value"});
  table2.AddRow(new string[] {
    "Hugo Reyes"});
  table2.AddRow(new string[] {
    "hreyes@dharmainitiative.com"});
  table2.AddRow(new string[] {
    "720-123-5477"});
  testRunner.Then("I should see the following details on screen:", 
    ((string)(null)), table2);
  testRunner.CollectScenarioErrors();
}

Once you’ve defined your first feature, Press Ctrl+R,T to run your SpecFlow tests. Your CreateCustomer test will fail as inconclusive because SpecFlow cannot find a matching step definition for the first step in your test (see Figure 6). Notice how the exception is reported in the actual .feature file, as opposed to the code-behind file.

Figure 6 SpecFlow Cannot Find a Step Definition

Because you haven’t yet created a Step Definition file, this exception is expected. Click OK on the exception dialog and look for the CreateABasicCustomerRecord test in the Visual Studio Test Results window. If a matching step isn’t found, SpecFlow uses your feature file to generate the code you need in your step definition file, which you can copy and use to begin implementing those steps.

In your AcceptanceTests project, create a step definition file using the SpecFlow Step Definition template and name it CreateCustomer.cs. Then copy the output from SpecFlow into the class. You’ll notice that each method is decorated with a SpecFlow attribute that designates the method as a Given, When or Then step, and provides the RegEx used to match the method to a step in the feature file.

Integrating WatiN for Browser Testing

Part of the goal with BDD is to create an automated test suite that exercises as much end-to-end system functionality as possible. Because I’m building an ASP.NET MVC application, I can use tools that help script the Web browser to interact with the site.

One such tool is WatiN, an open source library for automating Web browser testing. You can download WatiN from watin.org/ and add a reference to WatiN.Core to your Acceptance Tests project to use it.

The primary way you interact with WatiN is through a browser object—either IE() or FireFox(), depending on your browser of choice—that provides a public interface to control an instance of an installed browser. Because you need to walk the browser through several steps in a scenario, you need a way to pass the same browser object between steps in the step definition class. To handle this, I usually create a WebBrowser static class as part of my AcceptanceTests project, and use that class to work with the WatiN IE object and the ScenarioContext that SpecFlow uses to store state between steps in a scenario:

public static class WebBrowser {
  public static IE Current {
    get {
      if (!ScenarioContext.Current.ContainsKey("browser"))
        ScenarioContext.Current["browser"] = new IE();
      return ScenarioContext.Current["browser"] as IE;
    }
  }
}

The first step you’ll need to implement in CreateCustomer.cs is the Given step, which begins the test by logging the user into the site as an administrator:

[Given(@"I am logged into the site as an administrator")]
public void GivenIAmLoggedIntoTheSiteAsAnAdministrator() {
  WebBrowser.Current.GoTo(https://localhost:24613/Account/LogOn);
  WebBrowser.Current.TextField(Find.ByName("UserName")).TypeText("admin");
  WebBrowser.Current.TextField(Find.ByName("Password")).TypeText("pass123");
  WebBrowser.Current.Button(Find.ByValue("Log On")).Click();
  Assert.IsTrue(WebBrowser.Current.Link(Find.ByText("Log Off")).Exists);
}

Remember that the Given portion of a scenario is for setting up the context of the current test. With WatiN, you can have your test drive and interact with the browser to implement this step.

For this step, I use WatiN to open Internet Explorer, navigate to the Log On page of the site, fill out the User name and Password textboxes, and then click the Log On button on the screen. When I run the tests again, an Internet Explorer window will open automatically and I can observe WatiN in action as it interacts with the site, clicking links and entering text (see Figure 7).

Figure 7 The Browser on Autopilot with WatiN

The Given step will now pass and I’m a step closer to implementing the feature. SpecFlow will now fail on the first When step because the step is not yet implemented. You can implement it with the following code:

[When("I click the \" (.*)\" link")]
public void WhenIClickALinkNamed(string linkName) {
  var link = WebBrowser.Link(Find.ByText(linkName));
  if (!link.Exists)
    Assert.Fail(string.Format(
      "Could not find {0} link on the page", linkName));
  link.Click();
}

Now, when I run the tests again, they fail because WatiN cannot find a link with the text “Create New Customer” on the page. By simply adding a link with that text to the homepage, the next step will pass.

Sensing a pattern yet? SpecFlow encourages the same Red-Green-Refactor methodology that’s a staple of test-first development methods. The granularity of each step in a feature acts like virtual blinders for implementation, encouraging you to implement only the functionality that you need to make that step pass.

But what about TDD inside of the BDD process? I’m only working at the page level at this point, and I have yet to implement the functionality that actually creates the customer record. For the sake of brevity, let’s implement the rest of the steps now (see Figure 8).

Figure 8 Remaining Steps in the Step Definition

[When(@"I enter the following information")]
public void WhenIEnterTheFollowingInformation(Table table) {
  foreach (var tableRow in table.Rows) {
    var field = WebBrowser.TextField(
      Find.ByName(tableRow["Field"]));
    if (!field.Exists)
      Assert.Fail(string.Format(
        "Could not find {0} field on the page", field));
    field.TypeText(tableRow["Value"]);
  }
}
[When("I click the \"(.*)\" button")]
public void WhenIClickAButtonWithValue(string buttonValue) {
  var button = WebBrowser.Button(Find.ByValue(buttonValue));
  if (!button.Exists)
    Assert.Fail(string.Format(
      "Could not find {0} button on the page", buttonValue));
  button.Click();
}
[Then(@"I should see the following details on the screen:")]
public void ThenIShouldSeeTheFollowingDetailsOnTheScreen(
  Table table) {
  foreach (var tableRow in table.Rows) {
    var value = tableRow["Value"];
    Assert.IsTrue(WebBrowser.ContainsText(value),
      string.Format(
        "Could not find text {0} on the page", value));
  }
}

I re-run my tests, and things now fail because I don’t have a page to enter customer information. To allow customers to be created, I need a Create Customer View page. In order to deliver such a view in ASP.NET MVC, I need a CustomersController that delivers that view. I now need new code, which means I’m stepping from the outer loop of BDD and into the inner loop of TDD, as shown back in Figure 2.

The first step is to create a failing unit test.

Writing Unit Tests to Implement Steps

After creating a CustomerControllersTests test class in the UnitTest project, you need to create a test method that exercises the functionality to be exposed in the CustomersController. Specifically, you want to create a new instance of the Controller, call its Create method and ensure that you receive the proper View and Model in return:

[TestMethod]
public void GetCreateShouldReturnCustomerView() {
  var customersController = new CustomersController();
  var result = customersController.Create() as ViewResult;
  Assert.AreEqual("Create", result.ViewName);
  Assert.IsInstanceOfType(
    result.ViewData.Model, typeof(Customer));
}

This code doesn’t yet compile because you haven’t created CustomersController or its Create method. Upon creating that controller and an empty Create method, the code compiles and the test now fails, which is the desired next step. If you complete the Create method, the test now passes:

public ActionResult Create() {
  return View("Create", new Customer());
}

If you re-run the SpecFlow tests, you get a bit further, but the Feature still doesn’t pass. This time, the test will fail because you don’t have a Create.aspx view page. If you add it along with the proper fields as directed by the feature, you’ll move another step closer to a completed feature.

The outside-in process for implementing this Create functionality looks something like Figure 9.

Figure 9 Scenario-to-Unit Test Process

Those same steps will repeat themselves often in this process, and your speed in iterating over them will increase greatly over time, especially as you implement helper steps (clicking links and buttons, filling in forms and so on) in the AcceptanceTests project and get down to testing the key functionality in each scenario.

From a valid Create View, the Feature will now fill out the appropriate form fields and will attempt to submit the form. You can guess by now what happens next: The test will fail because you don’t yet have the logic needed to save the customer record.

Following the same process as before, create the test using the unit-test code shown earlier in Figure 3. After adding an empty Create method that accepts a customer object to allow this test to compile, you watch it fail, then complete the Create method like so:

[AcceptVerbs(HttpVerbs.Post)]
public ActionResult Create(Customer customer) {
  _repository.Create(customer);
  return View("Details", customer);
}

My Controller is just a controller, and the actual creation of the customer record belongs to a Repository object that has knowledge of a data storage mechanism. I’ve left that implementation out of this article for brevity, but it’s important to note that, in a real scenario, the need for a repository to save a customer should kick off another sub-loop of unit testing. When you need access to any collaborating object and that object does not yet exist, or doesn’t offer the functionality you require, you should follow the same unit test loop that you’re following for your Feature and Controllers.

Once you’ve implemented the Create method and have a working repository, you’ll need to create the Details View, which takes the new customer record and displays it on the page. Then you can run SpecFlow once more. Finally, after many TDD loops and sub-loops, you now have a passing feature that proves out some end-to-end functionality in your system.

Congratulations! You’ve now implemented a unit of end-to-end functionality with an acceptance test and a complete set of unit tests that will ensure the new functionality will continue to work as your system expands to add new features.

A Word About Refactoring

Hopefully, as you create unit-level tests in your UnitTests project, you’re constantly refactoring with each test creation. As you move back up the chain from passing unit tests to a passing acceptance test, you should follow the same process, watching for opportunities to refactor and refine your implementation for each feature and all the features that come after.

Be on the lookout for opportunities to refactor the code in your AcceptanceTests project as well. You’ll find that some steps tend to be repeated often across several features, especially your Given steps. With SpecFlow, you can easily move these steps into separate Step Definition files organized by function, such as LogInSteps.cs. This leaves your main Step Definition files clean and targeted at the unique scenario you’re specifying.

BDD is about focus in design and development. By elevating your focus from an object to a feature, you enable yourself and your team to design from the perspective of the user of a system. As feature design becomes unit design, be sure to author tests with your feature in mind, and ensure the tests are guided by discrete steps or tasks.

Like any other practice or discipline, BDD takes time to fit into your workflow. I encourage you to try it out for yourself, using any of the available tools, and see how it works over time. As you develop in this style, pay attention to the questions that BDD encourages you to ask. Constantly pause and look for ways to improve your practice and process, and collaborate with others on ideas for improvement. My hope is that, no matter your toolset, the study of BDD adds value and focus to your own software development practice.

Brandon Satrom works as a developer evangelist for Microsoft in Austin, Texas. He blogs at userinexperience.com and can be found on Twitter as @BrandonSatrom.

Thanks to the following technical experts for refviewing this article: Paul Rayner and Clark Sell