Article
01/21/2022

Test Driven Development: Top-Down or Bottom-Up?

Christian Johansen | March 1, 2011

Test-Driven Development is a programming process where unit tests are used to specify the behavior of the system prior to writing the actual implementation. TDD for JavaScript has been covered on ScriptJunkie previously by Elijah Manor. In this article I want to discuss the difference between bottom-up and top-down design in TDD.

Bottom-up design

Bottom-up is traditionally the most common way to apply test-driven development. Taking a bottom-up approach means to start with independent low-level objects and functions, such as a custom ajax interface built on top of a library like jQuery specifically designed for your application's backend resources, e.g. blogPost.get(3245, callback). The jQuery plugin for viewing Picasa's web albums Elijah built in the aforementioned article was built bottom-up. It started by defining the mechanics of the plugin and then adding to it, piece by piece until the plugin was capable of actually displaying images.

Top-down design

In top-down (or outside-in) design, you typically start all the way out in the UI and write tests that include user interaction. You then work from those tests, "going deeper" until you have a fully functional feature. Stubs or mocks are usually heavily employed, as the underlying interfaces do not yet exist. These fakes often also serve as the starting point for tests on the next level.

Top-down then bottom-up

The top-down approach to development is applauded by its proponents for its ability to have developers focus more tightly on value, and for the reduced risk of writing code that ends up being thrown away because it wasn't needed after all. Bottom-up on the other hand has its strength in resulting in loosely-coupled elements of reusable code. Bottom-up also requires a lot less stubs or mocks while in development.

Personally, I've always preferred bottom-up development. I like reusable code. I like it a lot. It makes perfect sense to my brain to identify discrete functionality, implement each one independently of the next and then linking them together to deliver greatness to the end-user. However, it's hard to argue against a practice which promises to increase my focus on business value of the code I'm writing. In this article I'll show you how you can sprinkle your development process with a bit of top-down to keep your priorities straight while chunking out reusable components in a bottom-up fashion.

Disclaimer: I'm not saying you can't achieve loosely-couple elements when developing top-down. What I am saying is that the bottom-up approach makes this a much more natural result.

Task at hand: Live search

To illustrate this mixed approach, I will show you examples from TDD-ing a live search plugin for jQuery. For those unfamiliar with "live search" (or "auto suggest/completion"), it is an enhancement we can apply to e.g. search forms. As the user types a list of suggested terms pop up beneath the text field, and items in this list can be selected to more easily find whatever one is looking for.

The first test: A high level overview

Starting off with a top-down approach, we will write a high-level functional test which describes the functionality from an end-user's perspective. To avoid too much implementation-specific mocking, this test will attempt to describe as closely as possible what our system does, and say as little as possible about how it's done.

First of all we need a test case and a test with a descriptive name. We can take advantage of the fact that JavaScript allows strings as property names and make the name a small sentence describing what we want to test.

TestCase("LiveSearchFunctionalTest", {
    "test should display suggestions as user types": function () {
    }
});

Adding some markup

Next, we need a form to enhance. I'll be running the tests with JsTestDriver, which does not use HTML fixture files to run tests, but rather generates them on the server. In most cases, doing away with the HTML fixture is a good thing, as it reduces the amount of ceremony necessary to write and run tests. However, it does mean we have to work a little harder to embed sample markup for the tests to manipulate.

To simplify the creation of sample markup for the tests, JsTestDriver provides a feature called "HTMLDoc", which allows us to embed some markup in a comment inside the test. The comment is parsed server-side by JsTestDriver and the resulting DOM nodes will either be assigned to a property on our test case (e.g. in-memory only, which is faster), or attached to the document. We don't want our live search plugin to assume it owns the document, so in-memory will do just fine:

TestCase("LiveSearchFunctionalTest", {
    "test should display suggestions as user types": function () {
        /*:DOC form = <form action="/search" method="get">
            <fieldset>
              <input type="text" name="q">
              <input type="submit" value="Go!">
            </fieldset>
          </form>*/

        // this.form now holds the form element
    }
});

Initializing the live search module

With the form in place we need to initialize our feature. This will be the only interaction with our API in this high-level test.

TestCase("LiveSearchFunctionalTest", {
    "test should display suggestions as user types": function () {
        /*:DOC form = <form action="/search" method="get">
            <fieldset>
              <input type="text" name="q">
              <input type="submit" value="Go!">
            </fieldset>
          </form>*/

        var form = jQuery(this.form);
        form.liveSearch();
    }
});

Simulating a user typing

Now that the module is all set up, we need to type in something and then assert that the right things happen. We'll only trigger the typing-related events that interest us, which in this case is the keyup event. Let's assume that our form searches a movie database, and we want to search for "Robocop". Because we don't want our server to handle individual requests for each and every keystroke, we'll require a minimum timeout before we actually hit the server. The resulting code looks like this:

var input = form.find("input[type=text]");

input.val("R");
input.trigger("keyup");
// Small delay, user cannot type instantaneously

input.val("Ro");
input.trigger("keyup");
// Small delay

input.val("Rob");
input.trigger("keyup");
// Small delay

input.val("Robo");
input.trigger("keyup");
// Small delay

input.val("Roboc");
input.trigger("keyup");
// Small delay

input.val("Roboco");
input.trigger("keyup");
// Small delay

input.val("Robocop");
input.trigger("keyup");
// Longer delay

This portion is pretty verbose. We can clean this up later, but until the test is passing, introducing complex logic will only increase the risk of bugs in the test.

Passing time

In the above snippet, there is one thing missing. While Robocop may be able to type at a sub-1ms rate, humans rarely do, and our test needs to reflect this. We also need the delays to verify that requests aren't fired until the minimum timeout has passed.

Rather than have the test actually wait after each keystroke, and thus be slow, we will use Sinon.JS to pass time artificially for us:

TestCase("LiveSearchFunctionalTest", sinon.testCase({
    "test should display suggestions as user types": function () {
        // Setup as before
        // ...

        input.val("R");
        input.trigger("keyup");
        this.clock.tick(67);

        input.val("Ro");
        input.trigger("keyup");
        this.clock.tick(98);

        // ...

        input.val("Robocop");
        input.trigger("keyup");
        this.clock.tick(150);
    }
}));

Wrapping the test case in a call to sinon.testCase enables the clock locally for each test. The clock.tick(ms) method makes sure any timers scheduled to run (with setTimeout or setInterval) within the next ms milliseconds is fired correctly. After the final keystroke we "wait" for 150ms, which is the desired timeout. At this point our code should hit the server.

Asserting that suggestions are displayed

When the server responds the live search plugin should render the suggestions as DOM nodes in the form. We will add some assertions that check for the expected number of suggestions as well as the ordering:

var results = form.find("ol.live-search-results");
assertEquals(??, results.length);
assertEquals("???", results.get(0).innerHTML);
// ...

This is the basis of the test verification. However, we don't have all the info yet, so it's impossible at this time to know how many results to expect, or what their content is. We will have to deal with the server request before we can answer this question.

Testing server communication

For reasons explained in my previous ScriptJunkie article on stubbing and mocking, actually connecting to a live server is not desirable inside the test. This means that we'll use Sinon.JS to help us test the server as well.

Fake server requests are handled in two passes with Sinon.JS. First we define what responses the server should know about. Secondly, we tell the server to respond to all requests at a convenient time. We can define the server resources at any point, but because it is part of the test's setup, it makes sense to group it with other setup code in the start of the test:

TestCase("LiveSearchFunctionalTest", sinon.testCase({
    "test should display suggestions as user types": function () {
        /*:DOC form = ... */

        this.server.respondWith(
            "GET", "/search?q=Robocop",
            [200, { "Content-Type": "application/json" },
             '["Robocop", "Robocop 2", "Robocop 3"]']
        );

        // ...
    }
}));

Here we tell the server how to handle GET requests for /search?q=Robocop, namely by responding with a JSON array of strings (i.e., movie titles). As soon as the user is finished typing and the minimum timeout of 150 milliseconds has passed, we instruct the server to respond to all requests by calling this.server.respond:

input.val("Robocop");
input.trigger("keyup");
this.clock.tick(150);

this.server.respond();

With this information so clearly available in the test, filling out the assertions is easy:

var results = form.find("ol.live-search-results");
assertEquals(3, results.length);
assertEquals("Robocop", results.get(0).innerHTML);
assertEquals("Robocop 2", results.get(1).innerHTML);
assertEquals("Robocop 3", results.get(2).innerHTML);

Requiring the timeout

There is one detail missing from our test so far. It appears that we are expecting there not to be any server requests until the user has "stopped" typing for at least 150 milliseconds. However, we have no assertions in the test that actually verifies this behavior. In order to enforce this requirement, we will add one more assertion right after the last keystroke, but before the final timeout, which asserts that no requests have been made so far:

input.val("Robocop");
input.trigger("keyup");
assertEquals(0, this.server.requests.length);

this.clock.tick(150);

And with that the test is completely, and reads like part of a user story on auto suggestions:

TestCase("LiveSearchFunctionalTest", sinon.testCase({
    "test should display suggestions as user types": function () {
        /*:DOC form = <form action="/search" method="get">
            <fieldset>
              <input type="text" name="q">
              <input type="submit" value="Go!">
            </fieldset>
          </form>*/

        var form = jQuery(this.form);
        form.liveSearch();
        var input = form.find("input[type=text]");

        input.val("R");
        input.trigger("keyup");
        this.clock.tick(67);

        input.val("Ro");
        input.trigger("keyup");
        this.clock.tick(98);

        input.val("Rob");
        input.trigger("keyup");
        this.clock.tick(69);

        input.val("Robo");
        input.trigger("keyup");
        this.clock.tick(103);

        input.val("Roboc");
        input.trigger("keyup");
        this.clock.tick(82);

        input.val("Roboco");
        input.trigger("keyup");
        this.clock.tick(112);

        input.val("Robocop");
        input.trigger("keyup");
        assertEquals(0, this.server.requests.length);

        this.clock.tick(150);
        this.server.respond();

        var results = form.find("ol.live-search-results");
        assertEquals(3, results.length);
        assertEquals("Robocop", results.get(0).innerHTML);
        assertEquals("Robocop 2", results.get(1).innerHTML);
        assertEquals("Robocop 3", results.get(2).innerHTML);
    }
}));

Running the test

To run the test you need a small configuration file for JsTestDriver, save it in jsTestDriver.conf:

server: https://localhost:4224

load:
  - test/*.js

The configuration file simply loads the test file. Start the JsTestDriver server, attach a browser and run the test. Unsurprisingly, it fails. At this point, reading the error messages from running the test will instruct you step-by-step in setting up the rest of the environment. When all errors are dealt with, you should have a configuration file that looks like the following:

server: https://localhost:4224

load:
  - lib/*.js
  - src/*.js
  - test/*.js

Where the lib directory contains Sinon.JS and jQuery and the src directory contains a single file, live_search.jquery.js with the following contents:

jQuery.fn.liveSearch = function () {};

At this point the test results in a failure rather than an error which means that the test is executing cleanly, but the assertions are not passing. This is not all that surprising as we haven't implemented the plugin yet. Passing this test cannot be achieved without taking a giant leap of faith, so we will shift focus.

Going bottom-up: Timeouts

The high-level functional test now provides us with a clear goal that has actual end-user value. Rather than continuing the top-down approach by assuming underlying interfaces through mocks, we will now build the required parts of the plugin bottom-up. The timeout and request queuing is an integral feature of the live search plugin and a good place to start.

Starting small: Queuing requests at the right time

As is usual when doing TDD, we will start with the simplest possible test we can imagine. The test will expect that no request is made immediately after a query is queued:

TestCase("LiveSearchTest", sinon.testCase({
    "test queuing query should not immediately send request": function () {
        var liveSearch = new LiveSearch();
        liveSearch.queue("Movie");

        assertEquals(0, this.server.requests.length);
    }
}));

Again we're using Sinon.JS to provide a fake server, and the test checks that the server saw 0 requests after calling the queue method of the yet non-existent LiveSearch object. The test fails, and passing it is a matter of defining the constructor and queue method (in src/live_search.js):

function LiveSearch() {}
LiveSearch.prototype.queue = function () {};

The test passes. With very little effort we have made progress, if ever so slightly. Next up, we'll expect a request to hit the server after the minimum delay:

"test should send request after minimum timeout": function {
    var liveSearch = new LiveSearch();
    liveSearch.queue("Movie");
    this.clock.tick(150);

    assertEquals(1, this.server.requests.length);
}

The test does not demand a lot from us. It expects a single request - any request - to have been sent to the server. Passing it is simple:

LiveSearch.prototype.queue = function () {
    setTimeout(function () {
        jQuery.ajax();
    }, 150);
};

While obviously incomplete, this implementation actually satisfies the test.

Tightening the focus on timeouts

The previous test hinted to us that something's afoot. As with the earlier functional test, adding meaningful assertions to the test proved difficult. Ideally, we'd like to check that the URL of the request contains the query string in some way. Unfortunately, it is not at all clear from the test what we expect the URL to look like, what the query parameter name (if any) should be, or even if the request is supposed to be a GET or POST.

One problem in the previous test is that it does not receive enough input to let us know how and where the request will be go. A bigger problem is that this lack of information is diverting us from the initial goal: modeling the timeouts. One possible conclusion is that the LiveSearch is currently expected to do two things: handle timeouts, and handle server requests. It will be easier for us to reason about timeouts if we can leave the server communication off to someone else. Thus, we will refactor what little code we now have, so as to delegate the network stuff to a collaborator:

TestCase("LiveSearchTest", sinon.testCase({
    setUp: function () {
        this.liveSearch = new LiveSearch();

        this.liveSearch.dataSource = {
            get: sinon.spy()
        };
    },

    "test queuing query should not immediately perform search": function () {
        this.liveSearch.queue("Movie");

        sinon.assert.notCalled(this.liveSearch.dataSource.get);
    },

    "test should perform search after minimum timeout": function {
        var ls = this.liveSearch;
        ls.onData = function () {};
        ls.queue("Movie");
        this.clock.tick(150);

        sinon.assert.calledOnce(ls.dataSource.get);
        sinon.assert.calledWith(ls.dataSource.get, "Movie", ls.onData);
    }
}));

The test now expects the liveSearch object to get its data through a collaborator called dataSource. This trivial change makes it very easy to verify that the query is used when requesting results. It also makes it easy to use the timeout logic with any data source - be it a jQuery.ajax backed one, a WebSockets one, or even a JSON-P based one. Also note the use of sinon.assert.* which behave just like built-in JsTestDriver assertions, except with better customized error messages for test spies.

In order to keep the tests passing, the implementation must be updated as well, and the final result looks like the following:

LiveSearch.prototype.queue = function (query) {
    var self = this;

    setTimeout(function () {
        self.dataSource.get(query, self.onData);
    }, 150);
};

Multiple queued queries

Having gotten the meddling requests out of our way, we can continue focusing on the timeouts. The next test will verify that making several queries extends the minimum delay required to actually request results:

"test should extend minimum timeout": function () {
    this.liveSearch.queue("Robocop");
    this.clock.tick(100);
    this.liveSearch.queue("Terminator");
    this.clock.tick(50);

    sinon.assert.notCalled(this.liveSearch.dataSource.get);
}

The test waits for a total of 150 milliseconds, but since only 50 milliseconds passed since the last query, we expect no request to have been made. The test fails as the "Robocop" query is still being passed to the data source. The solution is to clear any existing timeouts whenever we queue a query:

LiveSearch.prototype.queue = function (query) {
    if (this.timerId != null) {
        clearTimeout(this.timerId);
    }

    var self = this;

    this.timerId = setTimeout(function () {
        self.dataSource.get(query, self.onData);
    }, 150);
};

As the test now passes, we can add one more test that expects the most recent queued query to be the one shipped to the data source.

"test should discard old queued queries": function () {
    this.liveSearch.queue("Robocop");
    this.clock.tick(100);
    this.liveSearch.queue("Terminator");
    this.clock.tick(150);

    sinon.assert.calledWith(this.liveSearch.dataSource.get, "Terminator");
}

The test passes immediately, so we don't need to update the implementation. Note how the test does not check that the data source was not invoked twice - we already know it wasn't from previous tests. All we wanted to check at this point was that the request carried the correct query.

Straying from the happy path

So far we have only tested the happy path of the live search object. We will now write a test that expects no request to be made for empty queries:

"test should not make requests for empty queries": function () {
    this.liveSearch.queue("");
    this.clock.tick(150);

    sinon.assert.notCalled(this.liveSearch.dataSource.get);
}

With our current implementation of queue, this test fails. To pass it, we can abort the method if the query is empty:

LiveSearch.prototype.queue = function (query) {
    if (!query) {
        return;
    }

    if (this.timerId) {
        clearTimeout(this.timerId);
    }

    var self = this;

    this.timerId = setTimeout(function () {
        self.dataSource.get(query, self.onData);
    }, 150);
};

I have a feeling that it is not irrelevant where we abort the queue method. To investigate, we will write one last test that will expect previous queries to be aborted even if the new one is blank and does not trigger a server request:

"test should abort previous query even if new is empty": function () {
    this.liveSearch.queue("Robocop");
    this.liveSearch.queue("");
    this.clock.tick(150);

    sinon.assert.notCalled(this.liveSearch.dataSource.get);
}

My suspicions where right - this test is failing. To fix it, we just move the argument check past the timer cancellation:

LiveSearch.prototype.queue = function (query) {
    if (this.timerId) {
        clearTimeout(this.timerId);
    }

    if (!query) {
        return;
    }

    var self = this;

    this.timerId = setTimeout(function () {
        self.dataSource.get(query, self.onData);
    }, 150);
};

The data source and rendering

The LiveSearch instance is now capable of queuing requests the way we wanted it to, and we're free to pick a new task in order to eventually pass the functional test. Two more tasks remain: implementing a default data source, e.g. using XMLHttpRequest) and rendering results as DOM elements. I will leave TDD-ing these as exercises to you, and instead jump right to the finished plugin

The "finished" plugin

The following sample is the "finished" jQuery plugin. Note the quotes - this is only finished in the sense that it will pass our initial functional test.

jQuery.fn.liveSearch = function () {
    this.each(function () {
        var form = jQuery(this);
        var input = form.find("input[type=text]");
        var liveSearch = new LiveSearch();

        liveSearch.dataSource = new XHRDataSource({
            url: this.action,
            method: this.method,
            param: input.attr("name")
        });

        var renderer = new ListRenderer(this);
        liveSearch.onData = function (data) {
            renderer.render(data);
        };

        input.bind("keyup", function () {
            liveSearch.queue(this.value);
        });
    });
};

If you're curious as to how I chose my steps TDD-ing the two remaining objects, you can see the whole project on GitHub.

Refactored functional test

I mentioned early on that I did not want to reduce the verbosity of the functional test until it was passing. Well, now that it is we can take a shot at it. One way to reduce the verbosity is to add a helper function for typing a string into a text field. The helper can split the string into characters and fire the key event and tick the clock for each character entered. The final version of the functional test follows:

TestCase("LiveSearchFunctionalTest", sinon.testCase({
    setUp: function () {
        this.type = function(input, text) {
            var characters = text.split("");
            var str = "";
            
            for (var i = 0, l = characters.length; i < l; ++i) {
                str += characters[i];
                input.val(str);
                input.trigger("keyup");
                this.clock.tick(90);
            }
        }
    },

    "test should display suggestions as user types": function () {
        /*:DOC form = <form action="/search" method="get">
            <fieldset>
              <input type="text" name="q">
              <input type="submit" value="Go!">
            </fieldset>
          </form>*/

        this.server.respondWith(
            "GET", "/search?q=Robocop",
            [200, { "Content-Type": "application/json" },
             '["Robocop", "Robocop 2", "Robocop 3"]']
        );

        var form = jQuery(this.form);
        form.liveSearch();

        var input = form.find("input[type=text]");
        this.type(input, "Robocop", this.clock);
        assertEquals(0, this.server.requests.length);

        this.clock.tick(150);
        this.server.respond();

        results = form.find("ol.live-search-results li");
        assertEquals(3, results.length);
        assertEquals("Robocop", results.get(0).innerHTML);
        assertEquals("Robocop 2", results.get(1).innerHTML);
        assertEquals("Robocop 3", results.get(2).innerHTML);
    }
}));

Conclusion

In this article we have walked through TDD-ing a jQuery plugin. We started from the user's perspective, writing a functional test inspired by top-down design. We then proceeded to build one of the required objects in a standard bottom-up fashion. This process ensures we always keep our focus on the end-user and real business value, while still enabling us to do bottom-up TDD in the lower-level parts of our implementation.

This process is growing popular and the way it is described here approaches what is often referred to as ATDD - Acceptance test-driven development. Note that the functional test we started with is not necessarily an acceptance test as it only tests part of the feature (i.e. no keyboard navigation, no click handlers on the suggestions and so on), but still it illustrates how keeping end-user functionality in sight guides us in choosing the right things to implement, even when we're going all the way down.

About the Author

Originally a student in informatics, mathematics, and digital signal processing, Christian Johansen has spent his professional career specializing in web and front-end development with technologies such as JavaScript, CSS, and HTML using agile practices. A frequent open source contributor, he blogs about JavaScript, Ruby, and web development at cjohansen.no. Christian works at Gitorious.org, an open source Git hosting service.

Find Christian on:

Twitter - @cjno
Christian's Blog
Christian's Book - Test-Driven JavaScript Development