Manual Testing / QA Testing

  • Article

Applies to: Windows Communication Foundation

Published: June 2011

Author: Alex Culp

Referenced Image

This topic contains the following sections.

  • Defect Cycle
  • QA Environment
  • Summary
  • Additional Resources

Typically, the QA team has a talent for finding problems that developers miss, even when they have written thorough unit and integration tests. However, in many projects, web services are tested from the client applications only, with no tests to ensure that the services themselves behave as they should. Client applications only test that the client application works well. This does not validate that the services behave correctly, outside of the scope of the client application. Unfortunately, many QA teams do not have the technical expertise to use XML tools such as WCF Test Client, WCF Storm, or SoapUI. If the lifetimes of your services are going to exceed the single version of the client, make every effort to equip your QA team with the tools they need to test services directly.

Even with a qualified QA team, there is another problem. The test scripts that the QA team runs are not necessarily repeatable. While it is true that some of the test tools can save test scenarios and replay them, a better approach is to integrate the test tools more fully into the development environment, or to have a junior-level developer on the QA team help incorporate tests that they have created into the automated integration tests. This ensures the repeatability of tests on subsequent releases. It also enables the QA staff to focus on new challenges.

Defect Cycle

It is important to create a process to ensure that the defects that the QA team finds stay resolved. The following figure illustrates such a process. Even if you have your own procedures, you may also want to incorporate some of the aspects that are depicted here.

Referenced Image

Here are the steps.

  1. QA identifies a defect and enters the details into the defect-tracking tool. If possible, the description of the defect should include the XML that calls the service, the response that was returned, and any other relevant information. The more details that QA can supply, the easier it is for the developer to resolve the defect. Assuming that the defect is triaged and is a valid defect (defect triage is not included in the diagram), it next goes to the developer to be resolved.

  2. When a developer receives the defect report, he or she should first write a test to duplicate the defect. Unit tests are preferable to integration tests because they usually are easier to write, and typically run as part of a CI build. If your team uses Team Foundation Server to track defects, the developer should decorate the test with a WorkItem attribute to ensure that the test is associated with the defect.

  3. The developer fixes the defect. The unit (or integration test) should now pass.

  4. QA verifies that the defect is resolved and closes the defect report.

QA Environment

Most enterprises have a separate QA environment in which tests are performed. Typically, this environment more closely mirrors the production environment than the development environment, and it has greater control over the data. Generally, the QA team does not want any bad data from developer's integration tests in their databases (this is possible, even if the integration tests clean up after themselves). For this reason, automated tests that are created by the QA team are usually the only tests that are run in the QA environment.

Summary

If applied correctly, the techniques discussed in this article will greatly reduce the amount of time and resources your company spends to maintain its applications. However, because they are a significant investment, do not institute these tests unless they have the same importance as the application code. Many projects are begun with good intentions, and with good testing procedures, but when time and budget constraints come into play, these test became obsolete as they are neglected, and no longer run. If this happens, all the money that was invested in them is effectively wasted.

Additional Resources

For more information, see the following resources:

Previous article: Integration Testing

Continue on to the next article: Imperative vs. Declarative Services