Extreme ASP.NET
Tools of the Trade: Application Center Test
Rob Howard
Contents
Application Center Test
Using Application Center Test
Running a Test and Establishing the Baseline
Know What You're Improving
Setting Up a Test Environment
Next Steps and Test Coverage
Conclusion
When you sit down to write an ASP.NET application, how much time do you spend thinking about performance? It's unfortunate, but for most developers performance is an afterthought. Performance planning and design really need to be front and center. Think about your goals and make sure one of those goals is good performance. Then make sure to measure your progress. The more you measure, the more opportunity you have to improve performance.
In this month's column I'm going to walk you through one of the great tools included with Microsoft® Visual Studio® Enterprise Edition: Microsoft Application Center Test. It's one tool that serious Web developers must have in their utility belts.
Application Center Test
Prior to leaving Microsoft, I did a 12-city roadshow for ASP.NET. One of the segments covered performance and introduced many developers to Microsoft Application Center Test. The tool invariably generated a lot of interest and I got lots of questions about it.
You can find the Application Center Test tool as part of Application Center (just hunt through your old stack of MSDN® CDs or DVDs) or as part of your Visual Studio .NET Enterprise Edition installation at Visual Studio .NET 2003\Visual Studio .NET Enterprise Features. When you first open Application Center Test you are presented with a tree view for navigating the available Tests, Results, and Users. First, I want to show you how easy it is to create Tests.
Using Application Center Test
First, create a simple Web application. For example, I'll use the page shown in Figure 1 (note that I've used a little-known trick when writing code inline with an ASP.NET page: you don't have to write the full event declaration for Page_Load).
Figure 1 Simple Web App
<%@ Page Language="C#" %>
<%@ Import Namespace="System.Data " %>
<%@ Import Namespace="System.Data.SqlClient" %>
<%@ Import Namespace=" System.Configuration" %>
<script runat="server">
public void Page_Load() {
using(SqlConnection connection =
new SqlConnection(ConfigurationSettings.AppSettings["Northwind"]))
{
SqlCommand command = new SqlCommand(
"SELECT * FROM Products", connection);
connection.Open();
DataGrid1.DataSource = command.ExecuteReader();
DataGrid1.DataBind();
}
}
</script>
<form runat="server">
<asp:DataGrid id="DataGrid1" runat="server" />
</form>
The code in the figure, though not a recommended way to structure your application, is simple enough to perform some rudimentary tests on. Opening the page in a Web browser should return a populated DataGrid rendered as an HTML table.
Now that you know the page works, copy the URL to your clipboard, as you'll need it in a second. The example URL in my case is https://localhost/blackbelt/outputcache/test.aspx.
Next, navigate to Application Center Test, right-click on Tests and choose New Test. This opens the New Test Wizard Welcome screen. Click Next to choose the source for the new test and choose "Record a new test." Click Next again to select the Test Type, and then click Next again when prompted to select the Script Language (leaving the defaults), bringing you to the screen in Figure 2.
Figure 2** New Test Wizard **
Recording tests is what makes Application Center Test so easy to use. Click on "Start recording" and a new instance of Internet Explorer will open. Do not type a URL into the Address bar just yet (it should read about:blank). Instead, from within this new Internet Explorer instance select Tools | Internet Options and browse to the Connections tab. Next, click on the LAN Settings button. You should see something similar to the screen shot in Figure 3.
Figure 3** Connection Settings **
You will notice that the Proxy server settings are populated and are different from their normal values. That's because Application Center Test has opened a new instance of Internet Explorer and instructed it to use a private proxy server run by Application Center Test. Any requests made through Internet Explorer can be captured by the Application Center Test proxy, as you will soon see.
To complete the test, close the Internet Explorer dialogs and paste the URL from your test ASP.NET page into the Address bar. Clicking the green Go button in Internet Explorer or simply hitting Enter will render the DataGrid again. Next, close Internet Explorer and you should see something similar to Figure 4.
Figure 4** Captured Requests **
The request details portion of the dialog is now populated with the requests that the Application Center Test proxy has captured. These are the same HTTP requests sent by Internet Explorer. Now click "Stop recording," then Next. You will be prompted to give the test a name (I used "My Test") and you can then finish the wizard.
Congratulations you're now a performance testing jockey—pretty easy huh?
There are many other settings and configuration options for your new test. You can find these by right-clicking on the My Test node in the Tests list and selecting Properties. From there you can simulate multiple browsers, multiple users, parameters that include a warm-up period (where results are not reported), and the duration for the test. (You can explore these settings later and read some of the many other great articles that discuss the philosophy of testing and the best strategies for running your tests.) Instead of dwelling on the minutia, let's jump right into running the test.
Running a Test and Establishing the Baseline
To run the test you just created, simply right-click on your test and select Start Test. Once the test begins running, click on the Show Details button. The details box shows a graph of the running test as well as any errors that may be occurring during the execution of the test (see Figure 5). Running the test the first time establishes the baseline, which is used to measure against the current and future performance. In the figure, the graph shows the baseline at approximately 90 requests per second.
Figure 5** Baseline Graph **
Now that you have an established baseline you can make some changes to the application to either improve or degrade overall performance in order to gauge the effects of such changes. Granted, the test case I used is exceedingly simple, but I want to show you a few tweaks that you can make to this small piece of code that have a large effect on the performance of the app.
Know What You're Improving
The more you measure the more opportunity you have to improve. In this case I am going to make some small changes to the application and measure after each change. While this is not something you can realistically expect to do with every check-in, you should get in the habit of checking performance regularly. For my company's Community Server product, we have a baseline set of tests to measure the overall app performance. If major changes are made, a developer can investigate any performance improvement or regression using previous test data.
This first change I'm going to make to the sample application is to limit the amount of data returned. I'll do this by changing the SQL query from SELECT * FROM Products to SELECT TOP 25 * FROM Products. This may seem like a minor change to the code. After all, all I have done is limit the amount of data output to the screen, but the results are surprising. The performance jumped from ~90 RPS (requests per second) to more than 200—that's greater than a 100 percent gain! Because you had a baseline, you know that limiting the amount of data bound in the DataGrid definitely affects performance. I've still got some tweaks to make.
Next, modify the <asp:DataGrid/> server control by adding EnableViewState="false":
<asp:DataGrid EnableViewState="false" id="DataGrid1" runat="server" />
ViewState is a great feature of ASP.NET, but it is not always needed. In fact, in most cases where the DataGrid is used ViewState is not needed. By disabling ViewState, I was able to increase performance 166 percent above the baseline in my test case. Hold on, I'm not finished yet!
Next, add the following line to enable OutputCaching:
<%@ OutputCache Duration="60" VaryByParam="none" %>
Figure 6** Output Caching **
Now rerun the test. Surprised! When I did this, I experienced a performance gain of more than 600 percent over the baseline, as you can see in Figure 6. Now if you really want to have some fun, play with the OutputCache duration numbers. For example, try an OutputCache duration value of 1 second—you'll be surprised again by the results.
Setting Up a Test Environment
Tests are CPU-intensive, so when running a test you should expect to see your CPU running at nearly 100 percent. This takes away resources from the application you are testing as it is sharing CPU time with the test harness. All kinds of configurations will affect your test results, and some will simulate the real world better than others. For example, when running a test with both SQL Server™ and the ASP.NET application on the same machine, you are not simulating the cost of network I/O. Most applications use a database on a separate server from their Web server.
To set up a real test environment, dig up a couple of old developer machines to use as your clients. Do not use virtual machines. Run Application Center Test on these machines. Next try to simulate your real environment as much as possible. Run the Web application on a server operating system with no other applications running and connect to a database on a separate server.
Those caveats aside, there is nothing wrong with running smoke tests on your developer box. Smoke tests that don't simulate a real environment, such as the examples shown earlier, still provide lots of valuable data and often can be extrapolated to forecast what those same tests would look like in a real-world scenario.
Next Steps and Test Coverage
Now that you have an idea of how to test and measure I would recommend trying to use Application Center Test on your own applications. It's a good idea to find out how your users typically use your application. Which pages have to perform really well, and which pages really don't?
For example, in Community Server we have several different types of performance tests we run. Mainline tests include anonymous scenarios (when no user is logged in) as well as authenticated scenarios. In applications that do heavy personalization this can significantly change the performance characteristics.
Mainline tests also include common paths through the system, such as the main view for blogs, galleries, or forums as well as the various views for each screen. It's obviously important that these mainline scenarios perform very well and it's best to spend a lot of time here to ensure you get them right.
If you manage or run any project that must support two or more simultaneous users and you don't know how many requests per second your main pages can handle, you've got a problem.
Once you have your performance tests in place, measure after any significant milestone. If you've structured your code effectively you can always go into the source and comment out sections and recompile. This can help you determine if the problem is a poorly written routine or something else. The something else, nine times out of ten, is going to be your data access code. In a later column I'll detail two tools in SQL Server, Profiler and Query Analyzer, that can be used to isolate performance problems in the database—and you don't need to be a DBA to understand them.
You can also test common paths through the application. When recording your test, just follow some of the common navigation paths you expect a user to follow. Application Center Test will record those and you can replay that exact script. If you want to really get fancy you can edit the generated VBScript files to introduce delays or other meaningful inputs into your test script—that's a separate article though.
The last test I would recommend requires a bit more work. For example, in Community Server our developers wanted to test how many posts per minute the application can support. To test this, rather than attempting to post content to a form, we created a new ASP.NET page that used the APIs to post the content. This page was then run through Application Center Test and a benchmark was established for how many posts per second the application can support. In other words, sometimes you need to do a little bit of work to test all your scenarios.
Conclusion
I didn't cover everything that Application Center Test provides, but I hopefully did give you enough working knowledge of Application Center Test so that you can begin using it to measure your applications and improve them. Just remember, establish a baseline, measure often (at least after every major application change), and identify your key scenarios. By following these simple rules you'll develop a better understanding of your applications and hopefully identify opportunities to improve their performance. In my next column I'll introduce some debugging tips and tricks that I use when writing Web applications.
Send your questions and comments for Rob to xtrmasp@microsoft.com.
Rob Howard is a founder of Telligent Systems, specializing in high-performance Web apps and knowledge management and collaboration systems. Previously, Rob was employed by Microsoft where he helped design the infrastructure features of ASP.NET 1.0, 1.1, and 2.0. You can contact Rob at rhoward@telligentsystems.com.