Yugan Sikri

December 2007

Revised February 2008

Summary: This article discusses the process for building a robust and extensive end-to-end testing process for SOA-based and component-based systems. (11 printed pages)

Contents

Introduction
End-to-End Testing: Background
End-to-End Testing: Design and Implementation
End-to-End Testing Using Selenium
Conclusion
Critical-Thinking Questions
Further Study
Glossary

Introduction

Datum Corporation is a software vendor that has a business model of building value-added software services for human-resource departments. They have been using service-oriented architecture (SOA) to build multiple components that are related to employee benefits. One of the recent modules that the vendor has completed is an employee-payroll module. Human-resource service providers and HR Departments can license the services from the vendor, and extend it to build other value-added services. The components can be integrated with a large set of other services and platforms. This includes integration with existing authorization and authentication frameworks, as well as other vertical services such as a health-benefits module.

A large HR services provider has recently licensed the payroll module and plans to offer it to their customers in five strategic international locations. This will enable their customers to upgrade their payroll system for many self-service functions, while retaining their existing infrastructure of other HR services. Figure 1 shows the relationships between the payroll-module components, and how they integrate with other parts of the HR systems.

Figure 1. Relationships between payroll-module components

The integration team at the service provider has been assigned the task of customizing and integrating the payroll service for its customers. After completing the integration, the team is responsible for the end-to-end testing of the module, as well as its integration with existing HR modules.

SOA promotes loose coupling of components, coarse-grained services, and adoption of horizontal and vertical standards for message exchange. A comprehensive, automated, and maintainable testing methodology is critical for the successful integration testing and functional testing of a component-based software system. This article discusses the process for building a robust and extensive end-to-end testing process. It follows a three-step process of test design, test execution, and test-results analysis. The example test case that is used in the application is called "add an employee."

End-to-End Testing: Background

Software testing is an extensive and critical activity for providing quality and robust software. Testing methodologies can be categorized into two techniques. The first is white-box testing, such as unit testing. It relies on detailed knowledge of the code, and checks different conditions of components, classes, and their methods. The second is black-box testing, such as functional testing. It focuses on the functional characteristics of the system, instead of the technology or code that is used to implement it. Both techniques are used in varying degrees for different testing needs.

End-to-end testing refers to user-level testing of component-based systems. It verifies that the integrated component works correctly as part of the overall system, and that the existing components of the system work as before. End-to-end testing approaches the system from both the functional side and the architecture side, thus combining black-box and white-box techniques. The steps that are required to perform end-to-end testing include test design, test execution, and test-results analysis. The following section will discuss these steps in detail.

End-to-End Testing: Design and Implementation

The payroll package has been integrated in an existing system, which has Web-based access for HR managers. The payroll package and the HR benefits system have the following testing requirements as part of their integration:

· Existing HR features must be verified as they were before the integration.

· New payroll features for HR managers.

· New payroll features for self-service.

End-to-end testing enables the building of test cases that are organized as a hierarchy of user functions or component features. It has the following characteristics:

· Focus on the testing from the end user's point of view.

· Create a comprehensive test scenario that can be used to create multiple sets of test cases. In this case, the test cases will focus on regression testing of existing system features, and testing new features of the payroll system.

Figure 2. End-to-end testing-design framework

Test Design

Test design is a process of creating a comprehensive set of test cases, based on the user functionality and different conditions that can occur for that user functionality. As Figure 2 shows, the following are the steps that are followed in the test-design phase:

Build User Functions

· Break down the system features into a hierarchy of features.

· Define related groups of functions at the top level (for example: "add a new employee").

· Decompose the function, based on input variations, output variations, and different execution paths (for example: valid employee, existing employee, successful add, failure to add employee).

Figure 3. Template for organizing user functions

Figure 4. Partial test-case hierarchy for the user function of adding an employee

· Identify relationships between user functions. Some functions might be identical; others might be contained in another, and some might be independent.

· Construct a hierarchical structure of the user functions (refer to Figure 4).

Build Conditions

· Build a set of conditions that correspond to each of the hierarchical user functions.

· Conditions include sequence and timing conditions, environment conditions, and data conditions.

Use the User Functions and Conditions to Build Test Scenarios

· A combination of the user functions and the various conditions that can affect those user functions become a powerful combination for building test scenarios.

· Test scenarios are aggregated use cases that combine one or more of the user functions and conditions to create practical usage scenarios. For example, an HR manager will log on to the system, check existing employees, add a new employee, verify the addition, and then log out of the system.

· Test scenarios can incorporate conditional logic, loops, and dependencies on other user functions.

Figure 5. Two example test scenarios using the user functions

Use the Test Scenarios and User Functions to Create Test Cases

· Test cases are sets of tests that are organized in multiple layers and together build comprehensive testing data.

· Test cases typically combine the user functions, conditions, and test scenarios to create a comprehensive set of end-to-end cases.

End-to-End Testing Using Selenium

Selenium is an open-source testing tool that has an integrated developer environment (IDE), tools, and scripting commands to enable testing of Web-based systems. We use Selenium for its flexible model, elegant design, ease of use, and extensive features.

The Selenium IDE comes as a Firefox plug-in, and it enables recording of Web applications. Web flow, which corresponds to user functions, is recorded as a set of actions, accessors, and assertions. Actions are actual activities on the application, such as a button click or data entry. Accessors examine the state of the application and store the results in variables. Assertions verify that the state of the application conforms to what is expected. Figure 6 shows the Firefox plug-in IDE.

Figure 6. Selenium IDE for test recording, script update, and playback

The following steps can be used to build a maintainable test suite with multiple test scripts for the different scenarios:

· By using the Web activity recorder feature, test scripts are initially created per the test cases.

Figure 7. First demo of a sample Web application page used for test case of "add an employee"

Figure 8. Second demo of a sample Web application page used for test case of "add an employee"

Figure 9. Third demo of a sample Web application page used for test case of "add an employee"

· A test script is created in the form of an HTML table, with each row representing a command, target, and value. It records the actions that represent the test case.

· Use the Selenium commands to create accessors and assertions. Accessors capture the value of data elements on the Web page, based on ID, XPATH expressions, or CSS selector pattern.

Figure 10. Example accessor to capture the value of total employees dynamically

Figure 11. Snippet of the test-case file generated by Selenium

· Create related test cases that are similar to the captured test case. In the case that test data is captured in spreadsheets, a simple program can be used to build multiple test cases, using the test data.

· Test scripts are organized as test suites. A test suite is a list of references to multiple test cases.

· Use Selenium to run the test suite. The test suite executes each step, and displays the results against each step of the test. Tests can also be executed by using alternate tools or scripts.

Figure 12. Snippet of the Selenium test runner

Conclusion

End-to-end testing is a comprehensive testing process for systematically adding and testing new components in software systems, as well as end-user testing of the overall integrated system. It combines black-box testing and white-box testing techniques. The process starts with test design, which follows a methodology for creating test cases. The test cases are then used to build scripts that test the functional and nonfunctional features of the system. Test cases are organized in multiple test suites, based on the specific needs of the application. Test suites can also be executed as part of other automated processes, such as automated releases and testing.

Critical-Thinking Questions

· Which areas are complex to test in an SOA-based system, and why?

· How can end-to-end testing be included in the automated release process?

· How does access to multiple versions of Web services affect the end-to-end testing process?

Further Study

Beizer, Boris. Black-Box Testing: Techniques for Functional Testing of Software and Systems. New York: John Wiley & Sons, 1995.

Myers, Glenford J., Corey Sandler, Tom Badgett, and Todd M. Thomas. The Art of Software Testing. Second edition. John Wiley & Sons, 2004.

Selenium IDE: Selenium IDE. [Date unknown.] Open QA Web site. [Cited January 10, 2007.]

Selenium Reference. [Date unknown.] Open QA Web site. [Cited January 10, 2007.]

Tsai, W-T, X. Bai, R.A. Paul, W. Shao, and V. Agarwal. "End-to-End Integration Testing Design." Proceedings of the 25th International Computer Software & Applications Conference on Invigorating Software Development. New York: IEEE Computer Society, 2001, pp. 166-171.

Gao, Jerry Zeyu, Jacob H-S Tsao, and Ye Wu. Testing and Quality Assurance for Component-Based Software. Boston: Artech House, 2003.

Glossary

EAI—Enterprise application integration is the use of software and computer-system architectural principles to integrate a set of enterprise computer applications.

ESB—Enterprise service bus is an architecture style that uses a common messaging bus to integrate enterprise computer applications.

ReST—REpresentational State Transfer is a term that is used by Roy Thomas Fielding in his Ph.D. dissertation to describe an architecture style of networked systems.

RMI—The Java Remote Method Invocation API, or Java RMI, is a Java application-programming interface for performing the object equivalent of remote procedure calls.

SOA—Service-oriented architecture is a style of systems architecture. SOA enables the creation of applications by combining loosely coupled and interoperable services.

About the author

Yogi Sikri is a Certified Senior IT Solutions Architect at Genworth Financial, and is involved in the architecture and design of large-scale service-oriented architectures in J2EE and other technologies. He has been involved in the software industry since 1993, and has expertise in OOAD architecture, design and development, and systems integration. Yogi has conducted several training programs. He is a Sun Certified J2EE Enterprise Architect (Part 1) and an IBM Certified Solution Developer in XML technology, and he holds an MS in Administrative Studies from Missouri University.

This article was published in Skyscrapr, an online resource provided by Microsoft. To learn more about architecture and the architectural perspective, please visit skyscrapr.net.