How To: Use ACT to Test Performance and Scalability

Article
07/14/2010

Retired Content
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

Improving .NET Application Performance and Scalability

J.D. Meier, Srinath Vasireddy, Ashish Babbar, and Alex Mackman
Microsoft Corporation

May 2004

Related Links

Home Page for Improving .NET Application Performance and Scalability

Chapter 15, Measuring .NET Application Performance

Chapter 16, Testing .NET Application Performance

How To: Use ACT to Test Web Services Performance

Send feedback to Scale@microsoft.com

patterns & practices Library

Summary: This How To shows you how to use the Application Center Test (ACT) tool to perform load tests. It describes how to configure ACT and establish project settings, and how to manually modify ACT scripts to address common requirements, such as using ACT with applications that use view state and a variety of different authentication mechanisms.

What You Must Know
Summary of Steps
Step 1. Create an ACT Project
Step 2. Create a Test
Step 3. Set Properties for a Test
Step 4. Run the Test
Step 5. Analyze the Output
Monitoring Client Health
Common Tasks
Additional Resources

Applies To

Microsoft® Application Center Test (ACT)
Microsoft Visual Studio® .NET 2003 Enterprise Developer and Enterprise Architect editions

Overview

Application Center Test (ACT) is designed to stress test Web servers and analyze performance and scalability problems with Web applications. ACT simulates a large group of users by opening multiple connections to the server and rapidly sending HTTP requests.

ACT is installed with Visual Studio .NET 2003 Enterprise Developer or Enterprise Architect edition. Before beginning this How To, you must have one of these two versions of Visual Studio .NET installed.

You can use the tool directly within the Visual Studio .NET integrated development environment (IDE), though this method provides limited project configuration options. Full options are available when you use the stand-alone ACT application.

What You Must Know

When using ACT for stress and load testing, be aware of the following limitations:

With a single ACT client, you can only run one test at a time.
ACT is processor-intensive, which quickly stresses the client computer. This additional stress to the client computer can distort your results.
ACT supports only synchronous communication to the server.

If you need to run concurrent tests from the same client, you may consider downloading the Web Application Stress Tool (WAST). Note that WAST is not supported. You can download WAST from https://www.microsoft.com/downloads/details.aspx?FamilyID=e2c0585a-062a-439e-a67d-75a89aa36495&DisplayLang=en.

Summary of Steps

The following steps guide you through the process of creating and running a simple test using ACT to simulate the usage of your application by multiple users:

Create an ACT project.
Create a test.
Set test properties.
Run the test.
Analyze the output.

Step 1. Create an ACT Project

A project is a container for a test or multiple tests that make up a suite. You must have a project before creating a test.

To create a new ACT project

Start Microsoft Application Center Test from the Start menu by pointing to All Programs, Microsoft Visual Studio.NET 2003, Visual Studio .NET Enterprise Features, and then clicking Microsoft Application Center Test.
On the File menu, click New Project.
In the New Project dialog box, type MyTestProject in the Name box, and then click OK.

Changing the Log File Location

Properties for a project include the location of the ACT log file. The log file contains the output of any Test.Trace method call used in the test script. By default, ACT writes the log to the ACT installation folder.

To change the log file location

On the Actions menu, click Properties.
In the Properties dialog box, click Debugging.
In the Folder to save log to box, browse to a folder of your choice, and then click OK.

Step 2. Create a Test

You can create a test script manually, or you can have ACT record your interaction with a Web application. The record feature may require you to make changes to the script for some test scenarios.

Recording an ACT Test

To create a test file by using the Record a new test feature of the ACT tool, follow these steps.

To record an ACT test

On the Actions menu, click New Test.
In the Welcome dialog box, click Next.
In the Test Source dialog box, choose Record a new test, and then click Next.
In the Browser Record dialog box, click Start recording. A browser window opens.
Type the URL of the page you want to start the test from.
From this browser instance, perform the required operations on the test pages. For example, place an order for books after searching the site, adding books to a shopping cart, and supplying payment information.
Close the browser instance.
In the Browser Record dialog box, click Stop recording, and then click Next.
In the Test name box, type in a name for your test.
Click Next, and then click Finish.

**Note **Recording the test creates a test script that does not work with view state and multiple users. For more information about working with view state, see "View State" later in this How To.

Step 3. Set Properties for a Test

Before you run the load test against your application, you can modify the test features to reflect your test requirements. To set test properties, right-click the test and click Properties. Properties of the test that can be modified include the test load level, test duration, number of users, performance counters to be monitored, and reporting features.

Simultaneous Browser Connections (Test Load Level)

Simultaneous browser connections equate to the number of unique connections made to your Web server. The number of users should equal or exceed this value because each browser instance hits the page as a different user. If you need to simulate 20 concurrent connections to the application, set this value to 20.

Test Duration

You can choose to run your test for a specified amount of time or for a given number of iterations. Each has a different purpose:

Run test for a specific duration. Choose this option when you want load or stress test results for a given amount of time. Make sure that the test runs long enough to capture the minimum number of user operations. Generally, for applications where one full cycle of operations takes t1 amount of time, run the test for at least 2 * t1.
Run test a specified number of times. This option does not use warm-up time, and begins logging the test results right away. In general, choose this option for investigative tests rather than for load and stress tests.

Warm-up Time

The delay that occurs as objects are initialized and as your application reaches a steady state can skew test results. Use a warm-up time to allow initialization activity to stabilize before collecting performance data. You can determine an approximate warm-up time by watching the time it takes for the CPU utilization value to stabilize, or by looking at the requests per seconds of the previous test results before it reached a steady state.

Users (Users Tab)

The Users setting allows you to specify the users that the tool will use for the test run. You have two options:

You can have ACT automatically generate the users needed for the test run.

– or –

You can specify a User group, which has predefined users.

The number of users defined in the User group needs to be equal to or greater than the number of simultaneous browser connections, because each browser instance hits the page as a different user.

Creating New Users

If you need to create more users, you can do so through the ACT user interface.

To create new users

In the main window of the ACT tool, right–click the Users folder for your project and click Add to create a new user group. Specify an appropriate group name.
Select the user group for which you need to generate users, and, on the main Actions menu, click Generateusers. Specify the number of users and the required user details, and then click OK. ACT generates the specified number of users with the details supplied.
To import users from other data sources, select the user group for which you need to import users, and, on the main Actions menu, click Importusers. Select the data source file and follow the wizard instructions.

For Web applications that require specific user name and password combinations, you can create users in a user group, and then the group can be selected from this tab on the properties page.

Note that all iterations of the script use the same user unless the test script programmatically calls Test.GetNextUser. For more information about the Test object's methods, see the ACT Help file.

Step 4. Run the Test

The next step is to run the load test against your application.

To run the test

On the Actions menu, click Start Test.
Wait for the test to finish. You should see a live graph showing requests per second.
When the test is complete, click Close.
On the Actions menu, click View Latest Results. You can also click Results in the tree view.

Step 5. Analyze the Output

ACT produces output immediately and displays it in the Test Status window as your test runs. The Test Status window shows the time elapsed and time remaining, requests per second (RPS), and three types of errors: HTTP, DNS, and Socket. Before using the test results, you should investigate any errors and resolve them.

When a test is complete, you can view the latest results by right-clicking the test name from the project tree on the left pane and then clicking View Latest Results. The most commonly used performance measurements are displayed to the right of the test name.

Table 1: ACT Results

Item	Details
Test Run Name	The display name for the report. Right-click the report to rename it or delete it from the project.
Date Started	The date and time the test run started.
Total Run Time	The test run duration, in seconds.
Total Iterations	The number of times the test looped through the test script during the test run.
Total Requests	The total number of requests sent during the test run.
Connections	The number of simultaneous browser connections property value at the time the test run occurred.
Avg Requests/sec	The average number of requests sent, per second. The value is calculated with data collected over a one-second time period.
Avg Time to First Byte (msecs)	The average time between sending the request and receiving the first byte of the server response.
Avg Time to Last Byte (msecs)	The average time between sending the request and receiving the end of the server response.
HTTP Errors	The sum of all responses with result codes in the 400 – 499 and 500 – 599 ranges.
DNS Errors	The sum of all DNS errors.
Socket Errors	The sum of all socket connection errors.

Monitoring Client Health

Running ACT places a load on the client computer. Specifically, ACT places a load on the processor and uses memory. Monitor basic health indicators on the client to verify that the client computer is not overloaded, which could produce inaccurate results. Use the following System Monitor performance counters, shown in Table 2, to monitor display system health.

Table 2: Monitoring ACT Client Health

Counter	Details
Processor \ % Processor Time	Should not have sustained usage of more than 75%.
Memory \ Available Mbytes	Should not be less than 20–25% of client memory.

Common Tasks

You generally need to customize an ACT test script to configure it for your specific application requirements. The most common areas that require specific configuration include:

Think time
Authentication
Secure Sockets Layer (SSL)
ViewState
Cookies
Tracing
Web Services

Think Time

By default, ACT executes requests in your test as fast as the client computer is capable of sending them. The default behavior of the ACT client tests the raw throughput of your application but does not represent the real-world behavior of users. You need to use Test.Sleep to simulate user interaction. Inserting calls to Test.Sleep allows you to specify a delay, in milliseconds, between requests to simulate user reaction time. For example:

' 5 second delay between user steps:
Sub Step1()
  'Some work as a first step
End Sub
Call Test.Sleep(5000)
Sub step2()
  'Some work as second step
End Sub

You can also use Test.Sleep to simulate the time it takes to fill out a form. In this case, you need to insert sleep times after the initial page request, and before submitting the form.

Sub CreateLogin()
  ' request new account page
  Call Test.Sleep(5000)
  ' create login code
End Sub

Authentication

ACT supports anonymous, Integrated Windows, basic, digest, and Passport authentication mechanisms for test execution. If you use the record test feature to create test scripts, you need to modify the test scripts to support Integrated Windows and passport authentication. The discussion that follows describes how to do so.

Windows Authentication

ACT supports Integrated Windows authentication while testing, but you cannot record tests for a Web application that has the Integrated Authentication option enabled within Internet Information Services (IIS). The following steps show you how to work around this problem.

To record by using an application that uses Windows authentication

Enable basic authentication, in addition to Integrated Windows authentication, on your IIS server.
Record the ACT test script for testing your application. Provide the appropriate domain, user name, and passwords for your Web application when prompted by the tool.
Change the Web application configuration back to Integrated Windows authentication.
Comment out or delete the following line of each request in the test script, which was recorded in Step 2.
```
oHeaders.Add "Authorization", "Basic XYZ"
```
Set up the ACT users (using a separate user group if needed) with the proper domains, user names, and passwords that you want to simulate.

For each request, change the following line in the script:

oRequest.HTTPVersion = "HTTP/1.0"
to:
oRequest.HTTPVersion = "HTTP/1.1"

Run the ACT test script.

Passport

Microsoft .NET Passport is a suite of services that enable single sign-on for user authentication. Passport Web applications use Secure Sockets Layer (SSL); as a result, you cannot use ACT to record sessions against applications that use Passport authentication. Because the implementation of Passport can vary depending on the application design, you have to modify the test script manually.

For more information and code samples that show you how to modify test scripts for applications that implement Passport, see Performance Testing Microsoft .NET Web Applications, available from Microsoft Press® at https://www.microsoft.com/MSPress/books/5788.aspx.

Basic and Digest

You can record sessions against applications that use basic and digest authentication, without any test script modification. Your test script contains the relevant code, as shown below.

For basic authentication:

oHeaders.Add "Authorization", "Basic xyz="

For digest authentication:

oHeaders.Add "Authorization", "Digest username="+chr(34)+"domain\username"+_
chr(34)+", realm="+chr(34)+"MyApp"+chr(34)+", qop="+chr(34)+"auth"+_
chr(34)+", algorithm="+chr(34)+"MD5"+chr(34)+", uri="+chr(34)+_
"/MyDir/"+chr(34)+", nonce="+chr(34)+"xyz"+chr(34)+",_
nc=00000001, cnonce="+chr(34)+"xyz"+chr(34)+", response="_
+chr(34)+"xyz"+chr(34)

Anonymous

ACT automatically supports anonymous authentication for both the recording and the execution of your test script.

Secure Sockets Layer (SSL)

Recording an ACT test script with an SSL-enabled Web application is not supported. This is because ACT records the test by using a proxy, and the data is already encrypted when it reaches the proxy. To work around this issue, use the following steps.

To record an ACT test script with an SSL-enabled Web application

Disable SSL on the Web server and record the ACT test script.
After recording the test, modify the call to Test.CreateConnection in the test file. Specify port 443 instead of port 80, and set the bUseSSL parameter to true.

The basic syntax for Test.CreateConnection is:

MyConnection = Test.CreateConnection(strServer, lPort, bUseSSL)

The connection syntax without enabling SSL on the Web server is:

MyConnection = Test.CreateConnection("WebServerName", 80, False)

The connection syntax with SSL enabled on the Web server is:

MyConnection = Test.CreateConnection("WebServerName", 443, True)

View State

ASP.NET uses view state to maintain state passed between the browser and Web server by using hidden form elements on a page. ACT does not natively support ASP.NET view state. The view state information recorded in the ACT script is only valid for the user identity used to record the script. Therefore, errors occur if the view state is reused by a test script. To solve the problem, parse the response and encode the view state manually.

To use view state with ACT

Open the test script recorded by ACT that you want to modify. Your recorded script may look like the following, which is generated by the registration page for the IBuySpy sample application.

oRequest.Body = "__VIEWSTATE=dDwtMTMxMzQyNDA4ODs7bDxSZWdpc3RlckJ0bj"
oRequest.Body = oRequest.Body + "s%2BPh66%2F3Nf3WRINSe993%2B9pPGt%2B0je&Name=a&Emai"
oRequest.Body = oRequest.Body + "l=a@a.com&Password=a&ConfirmPassword=a&RegisterBtn"
oRequest.Body = oRequest.Body + ".x=0&RegisterBtn.y=0"

Replace the ACT view state GUID with a variable.

The view state variable is identified by the "__VIEWSTATE=" variable in the response body.

Replace this GUID with a variable. Make sure you remove the entire view state and replace it with the strViewState variable — some of it may be concatenated to the second line. After making the change for view state (and adding script to automatically use a new user for the registration process), the body of your POST should look like this:
```
oRequest.Body = "__VIEWSTATE=" & strViewState
oRequest.Body = oRequest.Body + &Name=a&Emai"
oRequest.Body = oRequest.Body + "l=a@a.com&Password=a&ConfirmPassword=a&RegisterBtn"
oRequest.Body = oRequest.Body + ".x=0&RegisterBtn.y=0"
```
At the top of the test, outside all subroutines, declare the variable used to represent the view state in the test script.
```
Dim strViewState
```
Declare the following variables at the beginning of the GET subroutine in the test script.
```
Dim Pos1, Pos2
```

Add the following snippet of code to the GET request. Add the code between the Begin and End view state parsing comments. Additional code in the subroutine is shown below to show you where to add these lines in the GET subroutine.

If (oResponse is Nothing) Then
  Test.Trace "Error: Failed to receive response for URL to " + "/StoreVBVS/register.aspx"
Else
  ' Begin viewstate parsing
  If InStr(oResponse.Body, "__VIEWSTATE") Then
    Pos1 = InStr(InStr(oResponse.Body, "__VIEWSTATE"), oResponse.Body, "value=")
    Pos2 = InStr(Pos1, oResponse.Body, ">")
    strViewState = Mid(oResponse.Body, Pos1 + 7, Pos2 - Pos1 - 10)
    ' Manually encode viewstates:
    ' Replace all occurrences of "+" with "%2B"
    strViewState = Replace(strViewState, "+", "%2B")
    ' Replace all occurrences of "=" with "%3D"
    strViewState = Replace(strViewState, "=", "%3D")
  End If
  ' End viewstate parsing
  strStatusCode = oResponse.ResultCode
 End If
   oConnection.Close
 End If
End Sub

Final Script

After you've made these view state modifications, your test script should look like the following.

Sub SendRequestxx()
  Dim strViewState
  Dim Pos1, Pos2

  If fEnableDelays = True then Test.Sleep (2333)
    Set oConnection = Test.CreateConnection("localhost", 80, false)

  If (oConnection is Nothing) Then
    Test.Trace "Error: Unable to create connection to localhost"
  Else
    Set oRequest = Test.CreateRequest
    oRequest.Path = "/StoreCS/register.aspx"
    oRequest.Verb = "GET"
    oRequest.HTTPVersion = "HTTP/1.0"
    set oHeaders = oRequest.Headers
    oHeaders.RemoveAll
    oHeaders.Add "Accept", "image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*"
    oHeaders.Add "Referer", "https://localhost/StoreCS/Login.aspx"
    oHeaders.Add "Accept-Language", "en-us"
    oHeaders.Add "User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322)"
    oHeaders.Add "Host", "(automatic)"
    oHeaders.Add "Cookie", "(automatic)"
    Set oResponse = oConnection.Send(oRequest)

    If (oResponse is Nothing) Then
      Test.Trace "Error: Failed to receive response for URL to " + "/StoreCS/register.aspx"
    Else

      ' Begin viewstate parsing
      If InStr(oResponse.Body, "__VIEWSTATE") Then
        Pos1 = InStr(InStr(oResponse.Body, "__VIEWSTATE"), oResponse.Body, "value=")
        Pos2 = InStr(Pos1, oResponse.Body, ">")
        strViewState = Mid(oResponse.Body, Pos1 + 7, Pos2 - Pos1 - 10)
        strViewState = Replace(strViewState, "+", "%2B")
        strViewState = Replace(strViewState, "=", "%3D")
      End If
      ' End viewstate parsing

      strStatusCode = oResponse.ResultCode
    End If

    oConnection.Close
  End If
End Sub

Cookies

Cookies are handled automatically. You can specify them by using the Headers.Add method as follows.

Headers.Add "Cookie", "(automatic)"

Tracing

You can enable tracing to troubleshoot problems. You either enable tracing on the project for all tests, or you enable tracing within a test. To enable tracing at the project level, right-click the ACT project to display the Properties dialog box, and then enable tracing options on the Debugging tab. To enable tracing within your test, specify Test.TraceLevel as follows.

Test.TraceLevel=-1 '-1 logs all information

Test.TraceLevel has the following options:

–1: Log all information
0: Disable logging
1: Log internal program information only
2: Log external information only from the Test.Trace method. This option is the default.

You can add custom messages to your trace output. To do so, use the Test.Trace method as follows.

Test.Trace("Just a test")   ' Just a test is a custom message

The default location for the trace log file is "%ProgramFiles%\Microsoft ACT\ACTTrace.log" on the test controller computer. The log is cycled when a new test is run. If you want to preserve the log, copy it to a different location.

**Note **Recorded scripts will capture errors a server sends.

If you build a script manually, insert your script in the following If statement to capture any error pages that are returned.

Set oResponse = oConnection.Send(oRequest)
If (oResponse is Nothing) Then
  Test.Trace "Error: Failed to receive response for URL to " + "/session.aspx"
End If
oConnection.Close

Web Services

For information about how to use ACT to test Web services, see "How To: Use ACT to Test Web Services" in the "How To" section of this guide.

Additional Resources

For more information, see the following resources:

Chapter 15, "Measuring .NET Application Performance"
Chapter 16, "Testing .NET Application Performance"
"How To: Use ACT to Test Web Services Performance" in the "How To" section of this guide
Performance Testing Microsoft .NET Web Applications, available from Microsoft Press at https://www.microsoft.com/MSPress/books/5788.aspx

Retired Content
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.