New Stuff
Resources for Your Developer Toolbox
Marnie Hutcheson
Contents
Not Just Another Automated Web Test Tool
Quick Takes on Testing
Book Review
Not Just Another Automated Web Test Tool
Tired of spending oodles of time trying to reproduce bugs so you can then fix them? RadViewSoftware Inc. has a new addition to their Web application function and load testing tools that captures everything that goes on at the client and the server in synchronized logs. When something breaks or just slows down, you get all the information you need in order to see what caused the problem, without having to recreate the bug. This approach lets you find and remove the really subtle, irritating, hard-to-reproduce bugs that usually appear during production.
RadView offers a suite of four tools: WebFT, an automated capture replay tool for functional testing; TestView Manager, for managing tests through a central console; WebLOAD, for load, performance, and scalability testing; and the newest tool, WebLOAD Analyzer, which uses patented black-box technology to tell you exactly what happened, so you can solve the problem without having to recreate it.
RadView's tools all use JavaScript as their primary language, so tests that you create in WebFT can be reused in WebLOAD, the performance test tool. You can also customize your scripts using ActiveX® controls or by using WebFTs own functions. Or, you can keep it very simple and just let WebFT capture your tests and track them by using the default settings. You can always cut, copy, and paste elements from one script to another.
WebFT gives you a visual environment for capturing and creating test scripts (RadView calls them agendas) that has test recording, editing, debugging, verification, and reporting features. It records everything from the UI, including think time, plus all the object information, HTML, and so on.
There's a wizard to help you test Web services. You select a Web Services Description Language (WSDL) file and pick the methods and parameters you want to use. The tool creates the test agenda for you. There are several database test commands you can include as well.
During capture, everything is recorded at the object level so that scripts can work with dynamic Web sites. You can update the object properties manually, or you can choose to ignore any properties for replay. You can parameterize data in your captured agendas using the parameterization wizard. You can also assign output and input parameters to your test elements, and build or link to DataSets for your particular test agendas.
Verification is dependent on object type so, for example, a link in a table will have Text, Link, Table cell, and Table verifications. If it is an image, it could have image verification as well. When you define a verification, you set the severity level of the error, and define what to do if it fails. You can assign verification at the object level, the page level (assigns verifications to every object on the page), or at the global level (assigns the verification to all items on all pages). For example, it is possible to verify the title of a page against a value stored for that page.
TestView Manager lets you create and schedule test-tree plans that are comprised of your WebFT test agendas. It is a test management console that lets you create and run sophisticated test plans across multiple platforms and browser types, using any number of different tests.
TestView Manager has the ability to create summary reports of your test plan executions, giving you result reports at several levels of detail. You can start at the highest summary level and dig down to the individual test scripts and even down to the object level on individual pages. You can watch your test scripts execute through the test tree, viewing the execution log and the application screens as they are processed. When execution is complete TestView displays the test report. You also get a report of all pass/fail results for each test tree.
RadView's load and performance testing tool, WebLOAD, lets you take the tests you created in WebFT and use them to set up tests to emulate different types of browser settings and functional testing under load.
One of the most aggravating realities of software development is that few testers and fewer developers have the time or the tools to do enough testing to discover the error trends in their systems. Functional tests touch the major points, but they don't simulate what happens when real users start using your system. There are lots of tools available to find the "ultimate load," but I'm talking about simulating normal day-to-day business while you are still in the test phase.
You have to be able to run many tests repeatedly (just as users will) to determine the frequency or probability of occurrence of the "hard" bugs. And you need the absolute unbiased view of an automated replay report to tell you things like, "10 percent of all logons are failing." Otherwise, the human tester will just shrug it off and try their test again because they aren't testing logon specifically, they are testing something else, and even if they were inclined to report the failure, they can't because they can't reproduce it.
WebLOAD fills this requirement very well. It generates load for a single script or many different scripts according to the choices you make in the WebLOAD Wizard. You can run several different tests at the same time in various proportions. Throttle control lets you ramp the load up and down, so that you can simulate actual usage trends. You can have your users execute the next statement together or set your timers to let you know how long it takes to get from here to there. Your tests can use dynamic data and you can simulate connection speeds. You can save your load choices as a load session template so you can run this load series again.
WebLOAD collects and records detailed information on application infrastructure and correlates this information with user transaction activity and traffic bursts. You can create any number of reports while the test is running or after your tests have finished so you can get just the view or information you need. There are many different views available in the console and you can display several time-slice graphs in the same view as shown in the screen shot, or you can overlay multiple test sessions to compare their differences.
Clicking on any point in the graph lets you view the underlying activities. WebLOAD Analyzer gives you a synchronized view of the virtual user and system-level activity. For example, if you double-click on a spike for a Web page in the WebLOAD Console, you can then see in Analyzer all the operations that were executed for that particular Web page. Analyzer can also display and compare configuration settings, such as COM objects, registry settings, operating systems, and services running between two servers, side-by-side.
WebLOAD Analyzer collects and records detailed information on systems and application infrastructure and correlates this information with user transaction activity and traffic bursts. So, besides showing you the 50 worst performing operations in the run, it also lets you do things such as compare application and system settings to pinpoint configuration errors. If you have a problem server, you can compare its whole configuration to the configuration of your gold machine.
Test tools are a dime a dozen, so why make such a large investment in this software? These tools help you simply remove bugs rather than spend lots of time trying to find those bugs first. You save time and money, improve the quality of your code, and remove hard bugs before your system ever goes to production.
Price: TestView Manager (1 seat): $1,295; WebFT (1 seat): 1,295; WebLOAD (50 virtual users): $6,395; WebLOAD Analyzer (one console, two black boxes): $17,195.
Quick Takes on Testing
Eventcorder Suite by CMS is a simple, inexpensive, basic capture replay toolset with automatic data input capability. It lets you automate routine tasks and perform simple or complex (scripted) automation, testing, and benchmarking.
Price: $49 per single-user license.
QA Wizard 3.1 by Seapine Software lets you perform automated functional testing of Windows® and Web-based applications. New features include support for the Microsoft® .NET Framework, script debugging, a Run Wizard, enhanced exception handling, and much more. QA Wizard uses a simple capture replay interface. It looks like this product is a good fit for not-so-technical function testing.
Price: $2,995 per single-user license.
ANTS Profiler 2.1 by Red Gate Software pinpoints slow code in .NET-based applications very quickly. It tracks memory managed by the common language runtime, breaking usage analysis into different objects and identifying each by name, size, and location. It generates snapshots of memory usage that show you the location of memory leaks.
Price: $295 per single-user license.
Book Review
Buffer Overflow Attacks: Detect, Exploit, Prevent by James C. Foster, Vitaly Osipov and Nish Bhalla (Syngress, 2005) is an excellent book both for providing new information and for a handy desk reference. The authors give you line by line explanations of buffer overflow attacks, stack overflows, heap corruption, and format string bugs in Windows, Linux, and FreeBSD.
Don't think that just because your system is well secured that there isn't someone crafting a designer worm just for you. This book gives examples and case studies.
Price: $34.95.
All prices were confirmed at press time and are subject to change.
Send your New Stuff to newstuff@microsoft.com.
Marnie Hutcheson is president of Internet Development Associates (Ideva), a firm in Ocala, Florida, that specializes in Internet and intranet Web application design and development. She has published technical papers and books on various computing topics. You can reach her at marnie@ideva.com.