Patterns in Practice
Design For Testability
Jeremy Miller
Contents
Don't adjust your dial; this is still a column about software design fundamentals. This time around I'd like to talk about testability, both as an important quality of your designs and as another tool you can use to create and think through designs. There is undeniable value in doing automated testing, but the question is whether the benefits of automated testing are worth the time and manpower investment. The benefits and costs of automated testing are greatly affected by design and architecture choices. In this column I will examine the design issues, patterns, and principles that can enhance testability.
This column does not assume that you will be using, or want to use, the design practice of Test Driven Development (TDD). I would hope that this column would be useful to anyone who is interested in adding more automated testing to their process or is just interested in software design in general.
The Value Proposition
Developers write code, but the real mission is to ship software. The effort to ship software is the sum of the time spent designing, coding, debugging, and testing the software. Lean programming has taught that you shouldn't be doing local optimization. Instead of only worrying about the coding effort, keep your eye on the total effort. The value proposition of automated testing is to reduce the total time of coding + debugging + testing. Your team will come out ahead as long as automated testing is saving more time in debugging and testing than it's costing in extra coding time.
In the United States, we say that the only certainties in life are death and taxes. Well, add another one. There will certainly be bugs in the code you and your team just wrote (and mine, too). Maybe it's because of a misunderstood requirement, a genuine programming mistake, an unforeseen edge case, or a third-party API that doesn't quite do what you thought it would do. Regardless of how those bugs got into your code, you've got to find those problems and get rid of them.
In the end, testability is all about creating rapid and effective feedback cycles in your development process to find problems in your code. It's an axiom in software development that problems are cheaper to fix the earlier they are detected. Since detecting and fixing errors is such a major part of the development effort, it's worth our time as developers to invest in feedback mechanisms to find errors more quickly. When comparing the relative ease of developing in one codebase compared to another, I would go so far as to say that the single most important factor is how short the feedback cycle is between writing a little code and getting some sort of feedback about whether the code you just wrote is working correctly.
What Is Testability?
I'm going to define testability as the quality of a software design that allows for automated testing in a cost-effective manner. The end goal of testability is to create rapid feedback cycles in your development process in order to find and eliminate flaws in your code. Here are a few aspects of testability:
Repeatable Automated tests must be repeatable. You must be able to measure an expected outcome for known inputs. You cannot do automated testing on systems that cannot be set up into a known state.
Easy to Write The tests should be mechanically easy to write. The equation is very simple: if it takes a lot of work to set up the inputs to an automated test for a unit of code, you aren't going to recoup your investment in writing that test.
Easy to Understand The tests should be readable and intention-revealing. Ideally, the automated tests that you write should also serve as a useful form of design and requirements documentation. In the real world, changing an existing codebase will at some point break the existing automated tests. When that time happens, it's advantageous for the automated tests to be intention-revealing to help developers understand why the test is broken in order to fix the broken tests. Automated testing coverage should make teams more confident in their ability to change code, not afraid of breaking inscrutable tests.
Fast The automated tests should run quickly. Again, the goal is to make the overall development process faster by establishing rapid feedback cycles to find problems. Slow-running tests will retard productivity.
These four points are directly related to design decisions, specifically to being mindful in regard to the classic design qualities of separation of concerns, cohesion, and coupling—things that you generally want anyway. That said, there are certain design patterns, such as inversion of control and the various forms of separated presentation, that can greatly enhance testability.
As in all things, there are some trade-offs. Many of these patterns are controversial because they don't quite fit the current direction of Microsoft tooling, are relatively new, or seem to negate the usefulness of some traditional .NET approaches. I'd like to use the rest of this column to present some sample scenarios and design patterns, and try to explain why these patterns are considered so important by teams that value TDD and automated testing.
Isolate the Ugly Stuff
Five years ago I considered myself to be strong in object-oriented design. Then I worked on a couple of projects that used TDD and learned firsthand just how little I really knew. Before going into these projects, I had read quite a bit about writing unit tests with NUnit that showed examples like this:
[TestFixture] public class HolyHandGrenadeTester { [Test] public void users_should_count_to_three_after_pulling_the_pin() { new HolyHandGrenade().TheNumberToCountTo().ShouldEqual(3); } }
Armed with these examples, I went into my first TDD project and immediately ran into trouble. My C# code was interacting with databases, Web services, the ASP.NET runtime, and external systems. Those things were a lot harder to test than code that runs completely inside a single CLR AppDomain.
I specifically remember having a lot of trouble writing unit tests for a little custom workflow subsystem. The workflow data was fairly complex and hierarchical, so I decided at the time to store the state of the workflow as XML data in an Oracle CLOB field. The workflow code itself knew to pull out the XML data from the database, work on it, update the XML, then invoke more ADO.NET code to save the changes back to the database. My code looked like the contrived example shown in Figure 1.
Figure 1 Workflow Code
public void ProcessWorkflowEvent( string newStatus, long workflowId) { // First, go get the XML out of the database OracleConnection connection = new OracleConnection(ProjectConfiguration.ConnectionString); OracleCommand command = connection.CreateCommand(); command.CommandType = CommandType.StoredProcedure; command.CommandText = "sp_that_returns_the_workflow_information"; // and set the necessary parameters command.ExecuteNonQuery(); string xml = (string) command.Parameters["@xml"].Value; XmlDocument document = new XmlDocument(); document.LoadXml(xml); // depending upon what the new status is, make changes // to the Xml document, decide whether or not to send an email, // then save the changed Xml document with another set // of ADO.NET calls }
Let's think about what you would have to do to write automated unit tests for this code. The only way to feed the test input into this data was to create a properly formed XML document and then put that XML into a database row. There was a lot of business logic that would change the state of the XML file, and therefore a lot of permutations of previous state and user input that I had to test.
As I said before, I struggled mightily when I tried to write unit tests for this code. My real issue was getting the business rules about the workflow state changes correct. The need to set up the XML data in the database was slowing me down. This is when I learned my first important lesson about testability: some things are just plain ugly to deal with in testing.
What I needed to do was isolate the ugly stuff. All I really mean by "ugly stuff" is any kind of code or infrastructure that is complicated or laborious or just plain inconvenient to get into a test harness, or that makes tests run very slowly. My partial list of ugly stuff would be:
- Database access. As most TDD practitioners will tell you, tests that involve a database will run an order of magnitude slower than tests that can run completely within a single CLR AppDomain. Setting up data in a database is much more time intensive than simply building objects in memory because of referential integrity and data constraints. When you test against the database, you will often find yourself adding setup data that has nothing to do with the actual test to the database just to satisfy referential integrity and value constraints.
- GUI technologies, such as Windows Presentation Foundation (WPF) or testing Web apps directly in the browser. It is possible to test the user interface itself, but it's a significant investment in time and the tests run even slower than database tests.
- Active Directory access.
- Web services. It's hard enough to set up a known state for testing your own code. Coordinating with a completely separate team to set up tests across two or more systems is more difficult. My advice is to make sure that the functionality of your Web services can be tested independent of the actual Web service protocols. For Web services external to your code, I strongly advise having your code depend on an abstracted interface for external Web services rather than use a Web service proxy class directly. This will enable you to replace the external Web service with your own stubbed implementation during internal testing.
- Configuration files.
Now, all of those things are important, and all of them should be tested at some point in time. The key, though, is to decouple as much of your core application as possible away from this infrastructure to at least make your core application code easy to test. I should be able to test my data access in isolation once, then write simple tests for the business logic without any concern for the database.
Back to my sample workflow subsystem. At the time I was struggling to get the business logic correct, but the XML data setup and data access was slowing me down. What I really needed to do was walk right up to the business logic code and work with it entirely in memory with quick-running tests. The first thing I should have done was to isolate the business logic away from the infrastructure and XML:
public class WorkflowState { public DateTime LastChanged { get; set; } public string CurrentStatus { get; set; } public string Priority { get; set; } // Other properties public void ChangeStatus(string newStatus) { // WorkflowState will update itself based on // the current state of the WorkflowState object // and the newStatus } }
Let's say that you have a business rule that says you should escalate the priority of the WorkflowState to Urgent if the customer is upset. Now the workflow business logic can be tested in isolation from the database infrastructure and the XML like this:
[TestFixture] public class WorkflowStateTester { [Test] public void the_priority_should_be_escalated_to_urgent_when_the_customer_is_angry() { WorkflowState state = new WorkflowState(){Priority = "Low", CurrentStatus = "NotStarted"}; state.ChangeStatus("CustomerIsVeryAngry"); state.Priority.ShouldEqual("Urgent!"); } }
The workflow business logic is easier to test, but what else did you get out of this change? You've made the business logic easier to write and understand because it's now independent from the data access infrastructure and the XML format. You now have a much greater ability to change the data access and the business logic independently.
Testability is extremely important in areas of the code that have a lot of variability, require many permutations of input to adequately test, or are volatile in regard to changing requirements.
In the original ProcessWorkflowEvent method shown in Figure 1, the act of loading and persisting the XML document to and from the database is basically the same no matter what is happening with the workflow business logic. I can write a small handful of tests against that persistence code and be confident that it works. It took many more test scenarios to adequately test all the code paths through the business logic. I could have saved myself quite a bit of effort on the original project if I'd been able to unit test the business logic in complete isolation.
Using Fakes to Establish Boundary Conditions
You might have noticed that the second version of the workflow logic isn't complete. It still needs to interact with the database and possibly an e-mail server as well. I would rewrite the original ProcessWorkflowEvent method to something like the method on a new WorkflowService class shown in Figure 2. Instead of doing the data access and sending e-mail directly in WorkflowService, I'll have it delegate to some new services named IRepository and IEmailGateway that I'll explain very soon.
Figure 2 Processing Workflow Events
public class ChangeWorkflowStatusMessage { public string NewStatus { get; set; } public long WorkflowId { get; set; } } public class WorkflowService { private readonly IEmailGateway _emailGateway; private readonly IRepository _repository; public WorkflowService(IEmailGateway emailGateway, IRepository repository) { _emailGateway = emailGateway; _repository = repository; } public void ChangeStatus(ChangeWorkflowStatusMessage message) { // Fetch the correct WorkflowState object from the database WorkflowState workflow = _repository.Find<WorkflowState>(message.WorkflowId); workflow.ChangeStatus(message.NewStatus); // email the owner of this workflow and tell them that something happened // if this is an Urgent item if (workflow.Priority == "Urgent") { // workflow.Owner is the user who is responsible for // this workflow _emailGateway.SendChangeStatusEmailTo(workflow, workflow.Owner); } // Save any changes to the WorkflowState object _repository.Save(workflow); } }
One of the prerequisites for automated testing is establishing known inputs to the test. To fully test the WorkflowService.ChangeStatus functionality, I need to set up test inputs that would work on WorkflowState objects that are and are not Urgent. I could set up the database for those test inputs on each test run, but there's an easier way. Let's just use a fake database that can be more easily controlled in testing scenarios by using the Repository pattern to mediate between the application's domain model and the data access services.
My team has consistently chosen to use a Repository pattern with an IRepository interface:
public interface Irepository { T Find<T>(long id) where T : Entity; void Delete<T>(T target); T[] Query<T>(Expression<System.Func<T, bool>> where); void Save<T>(T target); }
The real implementation of Repository uses NHibernate and LINQ for NHibernate to transfer information between the Entity objects and the database tables. In unit testing scenarios, all you really care about is that an Entity was saved, deleted, or available from this IRepository interface because that's all that the rest of the application knows about. For testing, you can create a known database state by using an implementation of IRepository (see Figure 3) that simply stores objects in memory and uses LINQ-to-Objects instead of LINQ for NHibernate for querying.
Figure 3 Fake Database from IRepository
public class InMemoryRepository : IRepository { private readonly Cache<Type, object> _types; private MockUnitOfWork _lastUnitOfWork; public InMemoryRepository() { _types = new Cache<Type, object>(type => { Type listType = typeof(List<>).MakeGenericType(type); return Activator.CreateInstance(listType); }); } private IList<T> listFor<T>() { return (IList<T>)_types.Get(typeof(T)); } public T Find<T>(long id) where T : Entity { return listFor<T>().FirstOrDefault(t => t.Id == id); } public void Delete<T>(T target) { listFor<T>().Remove(target); } public T[] Query<T>(Expression<Func<T, bool>> where) { var query = from item in listFor<T>() select item; return query.Where(where.Compile()).ToArray(); } public void Save<T>(T target) { listFor<T>().Add(target); } }
In unit tests, you construct objects that depend on IRepository with an instance of the InMemoryRepository that was set up with the desired data. Figure 4 shows a sample unit test that uses InMemoryRepository to simulate a database state.
Figure 4 Test that Simulates a Database State
[Test] public void find_the_user_by_user_name_and_password() { // Set up an InMemoryRepository with a User InMemoryRepository repository = new InMemoryRepository(); User user = new User(){UserId = "jeremy"}; user.ResetPassword("thePassword"); // the password is hashed repository.Save(user); // Construct a new instance of SecurityDataService using the // InMemoryRepository from above SecurityDataService service = new SecurityDataService(repository); service.Authenticate("jeremy", "wrong").ShouldBeFalse(); service.Authenticate("jeremy", "thePassword").ShouldBeTrue(); service.Authenticate("wrong", "thePassword").ShouldBeFalse(); service.Authenticate("wrong", "and wrong").ShouldBeFalse(); }
The sample tests the functionality of the SecurityDataService class (see Figure 5). Internally, it needs to query an IRepository for the existence of a user with a username and password.
Figure 5 Class to Be Tested
public class SecurityDataService : ISecurityDataService { private readonly IRepository _repository; // The IRepository dependency is set up via Constructor Injection // In our real application, Both the real NHibernateRepository // and the SecurityDataService would be constructed by // an Inversion of Control container public SecurityDataService(IRepository repository) { _repository = repository; } public bool Authenticate(string username, string password) { return _repository.FindBy<User>(u => u.UserId == username && u.Password == User.HashPassword(password)) != null; } }
In the real application, the SecurityDataService uses the NHibernate Repository object that accesses the database. But in testing you can happily substitute a fake Repository that can easily be controlled to more quickly simulate multiple test cases. You can only do this type of substitution with fake objects if the classes that use IRepository only depend on the declared interface of IRepository. If the classes that use IRepository depend even indirectly on the fact that the normal IRepository does NHibernate-specific things, you won't be able to cleanly substitute the fake object.
Fake objects, such as stubs or mocks, are a huge advantage for doing emergent or continuous design by allowing you to build, and even design, a system incrementally by standing in for services that you haven't yet created.
The Gateway Pattern
Almost every project I work on contains at least one feature that requires the application to send automated e-mail notifications to the user depending on some sort of action and state in the system. But I most definitely do not want my application code getting hung up with e-mail functionality.
I like to wrap up e-mail functionality by using the Gateway pattern. In the Gateway pattern, an object encapsulates access to an external system or resource. I want to pull every single piece of code it takes to send e-mails, including SMTP-type configuration, and hide it away from the rest of my code inside an IEmailGateway object, like this:
public class EmailMessage { public string Body { get; set; } public string[] To { get; set; } } public interface IEmailGateway { void SendChangeStatusEmailTo(WorkflowState workflow, User owner); }
There are a lot of existing solutions for sending e-mails from .NET code, so I'm not really that concerned about the actual act of sending the e-mail, but I'm very concerned about whether my application code decided to send an e-mail, to whom, and what the message body was. What I need to test in the WorkflowService is the simple fact that it did or did not choose to send an e-mail based on the final status of the WorkflowState object.
Separate Deciding from Doing
One of the best things you can do in your code is to treat performing an action and deciding to take an action as two separate responsibilities. In the case of WorkflowService, it is only responsible for deciding to send the e-mail:
// email the owner of this workflow and tell them // that something happened if this is an Urgent item if (workflow.Priority == "Urgent") { // workflow.Owner is the user who is responsible for // this workflow _emailGateway.SendChangeStatusEmailTo(workflow, workflow.Owner); }
You could just let the code send the e-mail to the right people, but it's generally pretty annoying to set up fake e-mail accounts or run around and ask people if they just got an e-mail from your system. Instead, use a mock object in place of the IEmailGateway just to check whether WorkflowService decided to send the e-mail notification (see Figure 6).
Figure 6 Testing whether E-Mail Was Sent
[Test] public void send_an_email_if_the_WorkflowState_is_urgent() { // setting up the WorkflowState var theOwner = new User(); var theWorkflowState = new WorkflowState{Priority = "Urgent", Id = 5, Owner = theOwner}; repository.Save(theWorkflowState); // Exercise the ChangeStatus() method service.ChangeStatus(new ChangeWorkflowStatusMessage(){ NewStatus = "Urgent", WorkflowId = 5}); // Verify that the change status email was sent emailGateway.AssertWasCalled(x => x.SendChangeStatusEmailTo(theWorkflowState, theOwner)); }
Separating the e-mail functionality into IEmailGateway made it easy to unit test the business logic, but did it give you anything else? Pulling all access to the actual e-mail sending code into a single class helps in the following ways:
- Prevent duplication in the code. Repetitive coding is required to find configuration data to set up the System.Mail.SmtpClient object to send the e-mail in .NET code.
- It makes the code that needs to send the e-mails somewhat simpler. WorkflowService just needs to say: send this e-mail to this user.
- You can change the e-mail infrastructure. What if you want to use a third-party tool instead of System.Mail.SmtpClient for e-mailing? What if you want extra logging for each e-mail, or want to implement a retry capability for failures or want to queue up e-mails to send in a batch? If you consistently use the same Gateway pattern interface for access to sending e-mails, you can add more robust infrastructure later without changing the classes that depend on IEmailGateway.
Small Tests before Big Tests
You want fast and accurate feedback loops on your code to flush out problems quickly. Any automated test should tell you that there is a problem when something doesn't work as expected, but a small, focused test can tell you exactly where something is wrong. When a big end-to-end test fails, you have a lot of different factors to consider, which often makes debugging harder. You still need the big end-to-end test, but my strong advice is to structure your application in such a way that you can write and test granular units of code first to flush out problems quickly before you attempt to execute end-to-end tests.
In the WorkflowService example, before attempting to run the actual WorkflowService from end to end, I would do the following tests:
- Write the WorkflowState object and test all of its business logic with various permutations of state.
- Write a couple of tests to prove that I can correctly save and load a WorkflowState object from the database.
- Write some interaction tests against the WorkflowService to make sure that it is correctly coordinating the IEmailGateway, IRepository, and WorkflowState objects.
- Inside the concrete EmailGateway class, I would write unit tests to verify that it is forming the correct e-mail contents for a given WorkflowState and User object independent of sending the e-mail.
Only once I am reasonably confident that every one of the pieces of WorkflowService function independently will I try to execute a full test of WorkflowService that uses the real database and sends e-mails.
Smaller tests are cheaper to create, easier to understand, faster to run, and much simpler to debug. It sounds like more work to write a bunch of unit tests than a single integrated test, but I've frequently found it more efficient to test small units of code before attempting to integrate those units in a bigger test.
The Big Picture
I would say that my focus on testability over the past five years has had more impact on how I approach software design than anything else. Granted, there are times when I have to go out of my way in a design for no other purpose than to make the code testable. In the end, though, I think that testability goes hand in hand with the classical definition of a good design.
Carefully considering the question, how can I test this in isolation? is yet another tool that will help you arrive at the classic design qualities of cohesion, coupling, and separation of concerns in your codebase.