Integration Testing

Applies to: Windows Communication Foundation

Published: June 2011

Author: Alex Culp

This topic contains the following sections.

• Why Use Integration Testing?
• Test Categories
• Data Setup and Tear Down
• String Tests
• Detecting WCF Configuration Errors
• Best Practices for Integration Testing

Why Use Integration Testing?

Unit tests enable you to catch many defects. However, a limitation of unit tests is their inability to detect errors that occur across external boundaries. For example, a bug in a stored procedure would not be caught by unit tests. This situation can include when there are databases, other services, or even flat files. It is important to create tests that validate this functionality, and this is why integration tests are important. They verify service methods when all of the dependencies are in place.

Test Categories

Integration tests should have their own test category. The following code creates a custom test category attribute that is similar to the attribute for unit testing.

IntegrationTest Attribute

public class IntegrationTestAttribute : TestCategoryBaseAttribute
{
    public override IList<string> TestCategories
    {
        get { return new List<string> { "IntegrationTest" }; }
    }
}

Data Setup and Tear Down

Most services interact with a database in some way. One of the most important functions of an effective integration test is to set up and tear down data (see Do Not Depend on Your Data Being There). You should also clean up any test data that your test creates, or the database quickly becomes polluted. There are several ways to set up and tear down data, and several places to execute the logic that performs these actions.

Options for Data Setup and Tear Down

Use Code - The first option is to write some code that performs the setup and tear-down logic. Technologies that you can use for this include ADO.NET, LINQ to SQL, and Entity Framework. Entity Framework is probably the easiest to use because you can return all of the data elements for a particular table. For example, you can use Entity Framework to create a customer in a customer table, return that entity to the integration test, and then use that same entity to delete itself when you are done.

Use DB Backup/Restore. The other way that you can set up and tear down data is to use scripts, or a database backup file that contains all of the data that is needed to execute your tests. You will still need to write select logic, or use Entity Framework to retrieve the test data. This approach does have the drawback that it is not portable. You will not be able to reuse your tests as you deploy from one environment to another. For example, there is a good chance that your QA team will be in the middle of executing their own tests, and will not be happy when you overwrite their test data.

Centralizing Data Setup

If you have multiple service projects, it is probably a good idea to centralize the data setup and tear-down logic into a single assembly. This is because it is likely that you will need to access this logic from all of the projects. A good example is an online shopping application, where you must set up and tear down customer data. Both the customer's shopping cart and the integration tests that check the logic that handles the shipping of purchases, might need the ability to create a customer.

String Tests

String tests verify complex business scenarios that require you to string multiple operation calls together to verify the piece of business functionality. One example of a string test might be: CreateCustomer => AddItemToBasket => PurchaseCart. These tests typically model a user interface workflow, and its interactions with services.

Detecting WCF Configuration Errors

Another best practice is to new up the service and call it directly. For more information, see New Up the Service. The disadvantage of this approach is that you do not detect critical errors in the WCF configuration that might prevent the service from running at all. For this reason, consider creating an additional test category named DeploymentTest. Create one test for each service to validate that the service, along with its dependencies, is functioning. One way to do this is to create a service operation named IsServiceHealthy. (This method might also be useful also in the production environment to validate on a periodic basis that the service still works.) The following code is an example of the IsServiceHealthy service operation.

Example Implementation of IsServiceHealthy

public bool IsServiceHealthy()
{
    try
    {
        using (var cnn = new SqlConnection(connectionString))
        {
            //Example DB logic
            cnn.Open();
            using (var cmd = cnn.CreateCommand())
            {
                cmd.CommandType = System.Data.CommandType.Text;
                cmd.CommandText = "select TOP(1) BookID from Book";
                cmd.ExecuteNonQuery();
            }
        }
    }
    catch (Exception ex)
    {
        //Make sure you write the exception somewhere for monitoring,
        // in production it will be good know what blew up.
        ExceptionHelper.HandleError(ex);
        return false;
    }          
    return true;
}

Example Deployment Test

[DeploymentTest]
[TestMethod]
public void MyService_IsHealthy()
{
    //assemble
    var target = ChannelFactory<IMyService>.CreateChannel(new BasicHttpBinding(), new EndpointAddress(myEndpointUri));

    //act
    var ret = target.IsServiceHealthy();

    //assert
    Assert.IsTrue(ret, "IsServiceHealthy did not return true, check event log for details on the error.");
}

Best Practices for Integration Testing

The following paragraphs describe some of the recommended best practices for creating and performing integration tests.

New Up the Service

Integration tests should new up a service rather than call the service over the deployed protocol. The reason for this is twofold. The first reason is that, unless you do this, you cannot easily gather code coverage metrics when you call your operations. The second reason is that it makes it easier to step into your code. This approach does require that you duplicate a great deal of the configuration from the service's web.config file in your integration test. For example, you must copy the connection strings. A solution to this problem is to use an MsBuild script to copy the configuration from the service project to the test project.

Create Only the Integration Tests You Need

While integration tests provide great value, they also take a great deal of work to set up properly, and a great deal of time to execute. When you create integration tests, consider creating only the integration tests necessary to test the "sunny day" scenarios (the default scenarios) of your operation. Create enough integration tests to validate that there are no critical or high-severity defects. The key point is that you want to maximize the return on investment (ROI).

Do Not Depend on Your Data Being There

Tests that depend on specific data should always create that data before they execute. If you assume that the data is there, the test becomes brittle. Undoubtedly, another developer or database administrator will delete or modify the test data and break the test.

Use Multiple Asserts Per Test

Because of their dependence on external resources, integration tests do not perform nearly as well as unit tests. A large number of integration tests causes a nightly build to run considerably slower. For this reason, it is acceptable, or even preferable, to consolidate integration tests. It is better to ignore the principle of one assert per test, and lose the level of detail about what is broken, than to have extremely long test execution times, and much more code to maintain.

Prefer Unit Tests Over Integration Tests

If you can chose between creating a unit test or an integration test to validate the same piece of functionality, always choose the unit test. Unit tests run faster than integration tests, and should be a part of the (CI) build.

Previous article: Unit Testing: Constructing the Service and Creating the Test

Continue on to the next article: Manual Testing / QA Testing