Scale Testing
Scale testing determines the capacity of a target environment that is running an application so that you can adjust the system for the expected throughput. Additionally, the test should simulate other expected loads that will run on the same production farm as your application. The developers of the Partner Portal application used scale testing when they deployed the application to a SharePoint farm. For an outline of this approach, see Plan for performance and capacity (Office SharePoint Server).
To identify and eliminate performance bugs, you should perform stress testing before you begin scale testing. The steps for both tests are similar. Additionally, you need to estimate the throughput of your system before you begin the stress test.
The developers of the Partner Portal application estimated the throughput required for the production scenario and used the scale test to prove that they could meet the requirement.
For scale testing, they used the mix scenario, which produces a realistic usage pattern for each of the test cases used during the load test. The mix represents the percentage of cases in which the scenario is used by the testing tool. The following illustration shows an example of the mix used for the Partner Portal load test.
Partner Portal load test scenario
.png "Ff648064.9b00fb9d-0e53-4016-b4fa-4af31463d6cf(en-us,PandP.10).png")
Finally, the developers selected the datasets to load into the reference implementation to produce a realistic, but high, load level. For example, in the Product Catalog scenario, they created a dataset of 100,000 products.
Estimating the Throughput
For scale testing, you need to determine the target throughput that you need to support in a production environment. The Partner Portal application uses the following table to determine the throughput in requests per hour. If a system is already in production, then you should use actual loads to determine an appropriate ratio for load tests. If you do not have an existing production system, you will need to make educated estimates about the usage you expect for each scenario. The estimates for the Partner Portal are based on projections for partners and usage.
The calculation of requests per second starts with an estimate of the number of active users per day. You then factor in the expected usage patterns for active users and a ratio for the highest number of users who would have active sessions at one time. The following are the estimates that were used for the Partner Portal application:
- The test scenario included 100 partners with 10 users each, for a total of 1000 users.
- It was predicted that up to 800 users would access the extranet in an 8-hour period.
- The partner usage mix was estimated to be the following:
- 10 percent: light users who generate 35 requests per hour
- 60 percent: typical users who generate 50 requests per hour
- 25 percent: medium users who generate 125 requests per hour
- 5 percent: heavy users who generate 250 requests per hour
- It was estimated that 80 percent of all partner users will be active on the Partner Portal at peak time.
This information was used to calculate the requests per hour (RPH) that the farm needed to support. The following table shows the calculation.
Light load 10% |
Typical load 70% |
Medium load 15% |
Heavy load 5% |
Total |
|
|---|---|---|---|---|---|
# of users |
80 |
480 |
200 |
40 |
|
Requests per hour per user |
35 |
50 |
125 |
250 |
|
Total requests per hour |
2,800 |
24,000 |
25,000 |
10,000 |
61,800 |
The next step was to divide the total requests per hour by 3600 to calculate the requests per second. Doing this produced 17 requests per second. Finally, the number of requests per second was multiplied by the highest percentage of active users on the site (80%). This produced a final anticipated load of approximately 14 requests per second.
After calculating the anticipated load, you need to find hardware that can support it. You might also decide to add capacity to ensure that your system can continue to operate if a single server fails.
Creating the Load Test Environment
The SharePoint performance test environment for the Partner Portal application is a private network that is isolated from the corporate domain and has a dedicated domain controller. This configuration allows you to control all traffic and server load and eliminates any external factors that might affect the environment. The following illustration shows the configuration that was used for scale testing.
SharePoint scale test environment
.png "Ff648064.71001513-a0cb-4ef3-b8d6-661aa9cc8701(en-us,PandP.10).png")
Analyzing Scale Test Performance Data
As with stress testing, performance analysis is an iterative process of setting up and running a number of load tests and examining the counters to discover patterns. The goal of scale testing is to determine system capacity for hardware resources. Therefore, scale testing might require that you change the number of agents or users to produce the targeted processor utilization. Forty percent is a typical target load for a production system. This load level can handle unanticipated peaks as well as system failures that might transfer the load to a different server in the farm. The stress tests for the Partner Portal application had durations that ranged from 10 minutes to 12 hours.
The following graph illustrates a problem that was encountered while running the scale tests.
Scale test graph
.png "Ff648064.a122d230-7ffa-4af4-a88c-f94a97cf029c(en-us,PandP.10).png")
After three hours, the system performed a major reset. The cause was isolated to an environment configuration error, resulting in a modified state from a previous series of scale tests. This illustrates that it is important to make sure that the system starts from a known initial state.
After scale testing is completed, you will have tuned and optimized the production system settings for your system, and will have a good understanding of the hardware requirements for your production configuration.
More Information
For more information, see the following resources: