Test
Build Quick and Easy UI Test Automation Suites with Visual Studio .NET
James McCaffrey
This article assumes you're familiar with Visual Studio .NET and C#
Level of Difficulty
1
2
3
SUMMARY
The .NET Framework provides a surprising new way to quickly and easily create user interface test automation. By using objects in the System.Reflection and System.Threading namespaces, you can write automated tests in minutes instead of hours. This article walks you through the building of a typical Windows-based application that will be used as the test subject. The author then runs through the creation of a C#-based test tool that simulates clicking the test app's UI controls and checks the application's state. After the tool is built, the author explains in detail how it works so you can modify and extend it for your own use.
Contents
I
f you've ever tried to write an automated test for user interfaces, then you know it's time consuming and tricky. But using reflection and the ThreadPool object in the Microsoft® .NET Framework, you can write powerful user interface test automation quickly and easily. In this article I'll walk you through the creation of a simple Windows®-based application and a small, powerful test tool that will showcase these .NET features.
The Problem
Suppose you're developing an application for Windows that has a standard user interface. Visual Studio® .NET and the .NET Framework make it easy to create buttons, menu items, and all sorts of other controls. Of course, during your development efforts you perform implicit manual testing of the user interface by checking basic functionality as you code. But suppose you want to write an automated test that will do a more thorough job of checking the user interface. If your product's design is stable, and if you have lots of time and resources, the solution would be to buy dedicated UI test software. While there are several good tools available, they do have drawbacks. They are relatively expensive, they often use a proprietary scripting language, and they tend to require a long time to redo scripts if the product changes significantly. For these reasons, this solution is not practical in many development environments. There must be a better way to do UI test automation.
With that thought in mind, I set out to create a UI test automation tool that allows you to create a test script in under 15 minutes with less than one page of code, can be used by novice testers, and uses only .NET functionality with no external dependencies.
After some experimentation I discovered that .NET does in fact provide the resources to create user interface test automation that meets all three design goals. Because the solution is quick to implement, you can afford to create test automation even when your product's design changes frequently. Because the solution is easy to understand, other people working with you can maintain your test automation code with little or no ramp-up time. And because the solution uses only native .NET code, there are no external dependencies to break the test automation.
The Application
Let's create a simple app that will serve as a basis for testing. Launch Visual Studio .NET and create a new C# Windows Application Project named MyWinApp. From the Toolbox, add three button controls, one textbox, and one listbox. Leave all the properties of the controls alone. Your design should look like Figure 1.
Figure 1 A Simple App
Double-click button1 to register an event handler for the button and then add the following code to print "Hello World" in textBox1:
private void button1_Click(object sender, System.EventArgs e)
{
textBox1.Text = "Hello World";
}
As you can see, button1_Click takes two parameters. These parameters will become important when I write the test automation that simulates clicking the button. The sender parameter represents the object that generates the associated event—in this case, button1. The EventArgs parameter represents additional information for the associated event. In the case of a button click, no additional information is necessary.
Double-click button2 and add code that displays the two-line message in listBox1:
private void button2_Click(object sender,
System.EventArgs e)
{
listBox1.Items.Add("Goodbye World");
listBox1.Items.Add("Come back again!");
}
Observe that manipulating the listBox1 field involves calling a method (Add) on a property (Items). When I automate, I'll need to access fields, properties, and methods.
Finally, double-click button3 and add the following code to erase any messages in textBox1 and listBox1:
private void button3_Click(object sender,
System.EventArgs e)
{
textBox1.Text = "";
listBox1.Items.Clear();
}
Build and run the application. The application has distinct states. The initial state is {textBox1 = "textBox1", listBox1 = (empty)}. The state in Figure 2 is {textBox1 = "Hello World", listBox1 = "Goodbye World / Come back again"}. When I write the test automation, I'll need to be able to check state.
Figure 2 button2 Clicked State
The Solution
Now let's create a simple but powerful automated test tool that will simulate clicks on the MyWinApp buttons and then check its state. Launch a new instance of Visual Studio .NET. Create a new C# Windows Application Project and name it MyWinAppTester. Using the Toolbox, add three button controls and two textbox controls. Adjust the text properties of button1, button2, and button3 to LaunchApp, Invoke Method, and Run Test, respectively.
The key to this user interface test automation is in the System.Reflection and System.Threading namespaces. Go to Code View and add two using statements to the six that were generated by Visual Studio .NET:
using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using System.Reflection;
using System.Threading;
Now comes the heart of the test tool. Add the code shown in Figure 3 to your project just below the Main method. Before I explain this code, let's make the test automation work. Double-click the LaunchApp button control to get to its click method, and double-click the Invoke Method button and the Run Test button. Add the code in Figure 4 to the click methods. You'll have to modify the path to MyWinApp.exe in button1_Click to reflect its location on your machine.
Figure 4 Test Tool, Button Action Code
// Launch MyWinApp
private void button1_Click(object sender, System.EventArgs e)
{
testAssembly = Assembly.LoadFrom("C:\\MSDN\\MyWinApp\\bin\\Debug\\MyWinApp.exe");
Type t = testAssembly.GetType("MyWinApp.Form1");
testForm = (Form)testAssembly.CreateInstance(t.FullName);
ThreadPool.QueueUserWorkItem(new WaitCallback(RunApp), testForm);
}
// Invoke Method
private void button2_Click(object sender, System.EventArgs e)
{
object[] p = {this, new EventArgs()};
string meth = this.textBox1.Text.ToString();
InvokeMethod(testForm, meth, p);
}
private void button3_Click(object sender, System.EventArgs e) // Run Test
{
object[] p = {this, new EventArgs()}; // arguments for a button click
bool pass = true;
for (int count = 1; count <= 3; ++count)
{
InvokeMethod(testForm, "button1_Click", p);
Thread.Sleep(1000);
if ( (string)(GetProperty(GetField(testForm,"textBox1"), "Text"))
!= "Hello World")
pass = false;
ListBox.ObjectCollection lboc = (ListBox.ObjectCollection)
GetProperty(GetField(testForm, "listBox1"), "Items");
if (lboc.Contains("Goodbye World") )
pass = false;
InvokeMethod(testForm, "button2_Click", p);
Thread.Sleep(1000);
// check that textBox1 has "Hello World" & listBox1 has "Goodbye
// World"
InvokeMethod(testForm, "button3_Click", p);
Thread.Sleep(1000);
// check to make sure that textBox1 and listBox1 are both empty
}
this.textBox2.Text = (pass) ? "Pass" : "Fail"; // display test result
}
Figure 3 Test Tool, Core Methods and Objects
Assembly testAssembly = null;
Form testForm = null;
BindingFlags flags = BindingFlags.Public | BindingFlags.NonPublic |
BindingFlags.Static | BindingFlags.Instance;
static void RunApp(object state) // Need this function to pass into
// WaitCallBack().
{
Application.Run((Form)state);
}
private void InvokeMethod(Form form, string methodName,
params object[] parms)
{
EventHandler eh = (EventHandler)Delegate.CreateDelegate(typeof
(EventHandler), form, methodName);
if (eh != null)
{
form.Invoke(eh, parms);
}
}
private object GetField(object obj, string fieldName)
{
Type t = obj.GetType();
FieldInfo fi = t.GetField(fieldName, flags);
return fi.GetValue(obj);
}
private object GetProperty(object obj, string propertyName)
{
Type t = obj.GetType();
PropertyInfo pi = t.GetProperty(propertyName, flags);
return pi.GetValue(obj, new object[0]);
}
Now you're ready to build the project and run it. Click the LaunchApp button and MyWinApp will be launched, as shown in Figure 5. If you click button1, button2, or button3 on MyWinApp, you'll notice that it functions normally. I can use the MyWinAppTester to invoke individual methods in MyWinApp. For example, in the textbox next to the Invoke Method button, type "button2_Click" (case sensitive) and then click the Invoke Method button. This will simulate clicking button2 on MyWinApp by calling its associated method.
Figure 5 Tester Launching the App
Now for the grand finale. Close MyWinApp, then relaunch it by clicking the LaunchApp button. Click the Run Test button on MyWinAppTester. You will see the actions of a simulated user clicking button1, button2, then button3, three times each in succession. The test checks the state of MyWinApp after each simulated click and finally displays a Pass or Fail message in the last textbox.
How it Works
The core method behind the technique is as follows:
private void InvokeMethod(Form form, string methodName,
params object[] parms)
{
EventHandler eh = (EventHandler)Delegate.CreateDelegate(typeof
(EventHandler), form, methodName);
if (eh != null)
{
form.Invoke(eh, parms);
}
}
This method is primarily a wrapper for the Invoke method of a Form object. The method requires an event handler that was created using CreateDelegate so that the existing thread is used instead of spawning a new thread. Another way to understand this method is to examine its parameters. To invoke a method, I need to know what form the method belongs to, the name of the method, and any parameters that the method requires. The flags variable has the following value:
BindingFlags flags = BindingFlags.Public | BindingFlags.NonPublic |
BindingFlags.Static | BindingFlags.Instance;
This is a filter required by several Reflection methods. Typically, the methods that you will be interested in are private instance methods, but because the | operator means "or", I will catch more than what is absolutely necessary.
Although not essential, it is very useful to have two helper methods that return fields and properties with a given name from a given object (see Figure 6). In each case, I get the type of the object, get its information, and then return the field or property. The GetValue call in GetProperty requires the new object[0] argument because properties can be optionally indexed.
Figure 6 GetField and GetProperty Helpers
private object GetField(object obj, string fieldName)
{
Type t = obj.GetType();
FieldInfo fi = t.GetField(fieldName, flags);
return fi.GetValue(obj);
}
private object GetProperty(object obj, string propertyName)
{
Type t = obj.GetType();
PropertyInfo pi = t.GetProperty(propertyName, flags);
return pi.GetValue(obj, new object[0]);
}
Launching the application to test uses two elegant .NET features, Reflection and the ThreadPool object:
Assembly testAssembly = null;
Form testForm = null;
// Launch MyWinApp
private void button1_Click(object sender, System.EventArgs e)
{
testAssembly =
Assembly.LoadFrom("C:\\MSDN\\MyWinApp\\bin\\Debug\\MyWinApp.exe");
Type t = testAssembly.GetType("MyWinApp.Form1");
testForm = (Form)testAssembly.CreateInstance(t.FullName);
ThreadPool.QueueUserWorkItem(new WaitCallback(RunApp), testForm);
}
Here I declare an Assembly object that represents the assembly containing the .exe I am going to test. (This hardcoded path could also be chosen from a File Open dialog.) Similarly, I declare a Form object that represents the Windows Form of the test application. To launch the test application, I first use Assembly.LoadFrom to get the test assembly. Then I use GetType to get the class of the application Form. Notice that I must use the full name of the form. These methods are part of the System.Reflection namespace.
The last line of the previous snippet does most of the work. The QueueUserWorkItem method creates a new thread of execution. By doing this instead of creating a new process, the test automation can communicate with the test application because the two threads are running in the same process. QueueUserWorkItem requires a WaitCallback delegate, which in turn requires a method to call (RunApp) and any parameters that the method requires (testForm). This line of code basically means execute the RunApp method using testForm as its argument.
Because of the structure of the WaitCallback delegate, I need to define the RunApp method, which is simply a wrapper around Application.Run:
// Need this function to pass into WaitCallback().
static void RunApp(object state)
{
Application.Run((Form)state);
}
Running the Automated Test
With all the parts in place, let's review the code that simulates user interface test automation (see Figure 7).
Figure 7 User Interface Test Automation
private void button3_Click(object sender, System.EventArgs e) // Run Test
{
object[] p = {this, new EventArgs()}; // arguments for a button click
bool pass = true;
for (int count = 1; count <= 3; ++count)
{
InvokeMethod(testForm, "button1_Click", p);
Thread.Sleep(1000);
if ( (string)(GetProperty(GetField(testForm,"textBox1"), "Text"))
!= "Hello World")
pass = false;
ListBox.ObjectCollection lboc = (ListBox.ObjectCollection)
GetProperty(GetField(testForm, "listBox1"), "Items");
if (lboc.Contains("Goodbye World") )
pass = false;
InvokeMethod(testForm, "button2_Click", p);
Thread.Sleep(1000);
// check that textBox1 has "Hello World" & listBox1 has
// "Goodbye World"
InvokeMethod(testForm, "button3_Click", p);
Thread.Sleep(1000);
// check to make sure that textBox1 and listBox1 are both empty
}
this.textBox2.Text = (pass) ? "Pass" : "Fail"; // display test result
}
Remember that if I want to call a button_Click method, I need to pass it an object and EventArgs. I start by creating variable p which is an array of objects that has these two items. Next, I assume that the test will pass by setting a Boolean variable. The code is contained in a for loop that arbitrarily executes three times.
First, I call the InvokeMethod helper that calls the method names button1_Click, which is part of testForm, and I pass p (a sender object and an EventArgs object) to button1_Click. In short, I simulate clicking button1.
Thread.Sleep(1000) is an arbitrary delay of 1,000 milliseconds (1 second). Now, I use the GetField and GetProperty helpers to get the Text property of the textBox1 field in the testForm, and check to see if it is "Hello World". In short, I check the state of the test application. Checking listBox1 is a bit trickier, as you saw earlier. First, I get the Items property and store it into variable lboc, and then use the Contains method to examine it and make sure that it does not have anything in it yet.
In a similar way, I simulate clicking button2 and check its state, and do the same for button3. If no incorrect states are found, the value of the pass variable remains true, otherwise the value of pass will have been set to false. The result is displayed as a string in the TestResult textbox.
Conclusion
The .NET Framework has many features that have improved the development experience. A byproduct of these features is a powerful new way to write test automation. The technique presented in this article is a practical way to automate user interface testing in Windows. There are always costs involved in creating test automation, but because this technique is so easy to implement, it is ideal when product design is relatively dynamic or when you have limited resources for test automation.
For background information see:
HOW TO: Submit a Work Item to the Thread Pool by Using Visual C# .NET
Visual Basic .NET Code Sample: Reflection
James McCaffreyworks for Volt Information Sciences Inc., where he manages technical training engineers working at Microsoft. He has worked on several Microsoft products, including Internet Explorer and MSN Search. James can be reached at
jmccaffrey@volt.com or
v-jammc@microsoft.com.