Performance Counters in Load Tests on Windows Azure
When performing load tests, Visual studio enables you to collect data from performance counters (normally viewed using Perfmon.exe). Performance counters are specific to each machine (worker role or web role) hosting a tested system. Using Perfmon, you can specify processes to monitor, including application processes. This topic walks through the steps to record those counters while performing a Visual Studio load test on Windows Azure.
Note |
|---|
| When logging performance counters as shown in this article, the network boundaries reduce the throughput of messages that can be received. For example, in an on-premises scenario, logging many counters at a frequency of one per second is feasible. In the scenario described here, network latencies prevent equivalent performance. In practice therefore, reduce the number of counters you collect. Also, set the frequency of collection to 5 seconds or higher. For more information about creating counter sets, see Creating Performance Counter Sets. |
This topic is a variation of the basic scenario described in Using Visual Studio Load Tests in Windows Azure Roles. That set of articles describes how to run Visual Studio load tests on Windows Azure. In that basic scenario, a load test project generates traffic against a Windows Azure application, and all roles are in the same Windows Azure Connect group. In this topic, the worker roles and the system under test run are in the same Connect group, but run in different hosted services.
Authors: Sidney Higa, and Paolo Salvatori.
Acknowledgements: Jaime Alva Bravo.
Prerequisites
The following are required to enable the capture of performance counters.
-
Windows Azure Connect endpoint installation executable, Wacendpointpackage.exe.
See How to Install Local Endpoints with Windows Azure Connect. Download the executable for later use. -
Windows Azure endpoint activation token.
Follow the instructions in How to Install Local Endpoints with Windows Azure Connect to get the token. Save the token for later retrieval.
Overview
Visual Studio load tests are composed of two logical parts:
-
Load Test Components. The components include the test controller and the agents.
-
The System Under Test. The application subjected to load testing.
The system under test and the load test components can be deployed together in one Windows Azure hosted system. In that scenario, communication between the parts is optimal as the agent nodes can communicate directly with the role instances that run the application. The parts can be deployed in separate hosted systems by using Visual Studio tools, but performance counters cannot be collected. (This is explained later).
In this scenario, the system under test and the load test components run in different hosted services, but performance counters must be collected. There are two hosted services, one for the test controller and agents, and a second for the system under test. However, there is still a single Windows Azure Connect group used to enable communication.
The default method for creating a Connect group is to use Visual Studio, by using the Activate Windows Azure Connect checkbox. (For more information, see Using Windows Azure Connect to Create Virtual Networks.) In this scenario, the agents and the system under test run in different services. Thus the roles cannot communicate to each other using the default method—using the checkbox does not work. By design, Windows Azure Connect does not allow communication between role instances that run in different hosted services. The workaround is to install the Connect endpoint on:
-
The worker roles that host the agents.
-
The Web and/or worker roles that host the system under test.
Since you do not use the checkbox, the endpoint must be automatically installed through another process, as described here.
Installing the Connect Endpoint Automatically
To enable communication between hosted services, first download the Wacendpointpackage.exe. Store the executable with the Visual Studio project that contains the worker role that hosts the agent. Then configure the worker role to install the endpoint automatically, as shown.
To install a Connect endpoint automatically
-
Copy and paste the Wacendpointpackage.exe into the Visual Studio project for the worker role that hosts an agent. Follow these next steps to configure the role to install the endpoint automatically.
-
Run Visual Studio as Administrator, and open the solution that contains the Load Test project.
-
In Solution Explorer, paste Wacendpointpackage.exe into the Visual Studio project that contains the worker role hosting the agent.
-
In Solution Explorer, right-click the Wacendpointpackage.exe and click Properties.
-
In the Properties pane, set the Build Action property to None.
-
In the Properties pane, ensure that the Copy to Out Directory property is set to Copy always.
-
Run Visual Studio as Administrator, and open the solution that contains the Load Test project.
-
Create a batch file that runs the installation executable.
-
Create a file named Startup.cmd.
-
Copy and paste the following code into the file. Replace <windows azure connect group activation token> with the activation token for your Connect group.
Wacendpointpackage.exe /i /s /m en-us /token <windows azure connect group activation token>
-
Create a file named Startup.cmd.
-
Copy and paste the Startup.cmd file into the Visual Studio project for the worker role that hosts the agent. Set the file's Build Action property to None. Also set the file's Copy to Out Directory property to Copy always.
-
Configure the worker or web role to run the batch file.
-
Open the Azure Cloud project that contains the service definition and configuration files.
-
Open the ServiceDefinition.csdef file.
-
Under the <WebRole> element that hosts the agent, add the <Runtime> element and set the execution context to "elevated." Below the <Runtime> element, add the <Startup> element. Then add the <Task> element to run the batch file in elevated mode. The complete code is shown:
<Runtime executionContext="elevated"/> <Startup> <Task commandLine="setup\startup.cmd" executionContext="elevated" taskType="simple" /> </Startup>
-
Open the Azure Cloud project that contains the service definition and configuration files.
The described technique is commonly used for installing software on Azure worker and web roles. For more information, see Starting Tasks Before Role Instances Tart in Windows Azure.
Initialization
Before continuing to the next section, deploy and run the virtual machines on Windows Azure. The running instances are required to create the counter sets, and to enable mapping of counter sets. For more information about deploying and running the hosted services on Azure, see Staging an Application in Windows Azure.
Creating Performance Counter Sets
To create performance sets, use the Visual Studio Load Test tools.
To create a performance counter set
-
Open the Visual Studio project that contains the Load Test components.
-
Open the test project, and open the load test (*.loadtest) file.
-
Right-click the Counter Sets folder, and click Add Custom Counter Set. The default name is Custom1. Rename the counter set folder, as needed.
-
Right-click the new custom counter folder and click Add Counters.
-
In the Pick Performance Counters dialog, from the Computer drop-drown list select the computer that contains the system under test. The computer can be a role instance that is running on Windows Azure.
-
From the Performance category drop-down, select the performance counter to monitor, and click OK.
-
Repeat the previous steps to add the counters you require.
After creating the counter set, create a mapping for each role that is running.
To create a performance counter mapping
-
In the load test file (*.loadtest), open the Run Settings folder.
-
Under the Counter Set Mappings folder, find the names of the running Azure instances. The virtual machines have a name that typically begins with "RD." There is one virtual machine for each instance of the Agent worker role.
-
Right-click the machine node, and click Manage Counter Sets.
-
In the Manage Counter Sets dialog box, select the custom counter set to add it.
The custom counter set is now mapped to each worker role that runs an agent.
Enabling Pass-Through Security
Remote Desktop is the standard way to access a role instance. You can configure roles to use Remote Desktop with the Visual Studio tools. The Visual Studio tool creates a certificate with a username and password you specify. For more information, see Using Remote Desktop with Windows Azure Roles. When you employ this method, the username is automatically added to the Administrator group on all roles that are configured to use Remote Desktop. The same user must be part of the Administrators group on the worker role machines. That means, the same user name and password is used to log onto the agents as well as the system under test. The way to accomplish this is to create a local user, and use that identity to deploy the Load Test hosted service. To check whether the configuration is correct, use Remote Desktop to examine the Administrators group on a running worker role. Check to see that the same user is part of the group. For more information about creating the local user for the Load Test, see "Creating a Local User To Manage a Load Test Controller" in the topic Publishing the Load Test To Windows Azure.
Build Date:
Note