[This documentation is for preview only, and is subject to change in later releases. Blank topics are included as placeholders.]
Test settings use diagnostic data adapters, which specify various types of data to collect or how to affect the test machine when you run manual tests, automated tests, or both. For example, a diagnostic data adapter might create an action recording, an action log, a video recording, or collect system information. Additionally, diagnostic data adapters can be used to simulate potential bottlenecks on the test machine or reduce the available system memory. For example, you can emulate a slow network to impose a bottleneck on the system.
Test settings define the following:
The type of tests you will run (manual or automated)
The set of roles that are required for your application under test
The role to use to run your tests
The diagnostic data adapters to use for each role
.gif) Note |
|---|
When you create test settings for your tests in Microsoft Visual Studio 2010, the only type of tests that you can run are automated tests. Therefore, you cannot select the type of tests to run. |
Running Tests Using Microsoft Test and Lab Manager
If you run your tests using Microsoft Test and Lab Manager, you can configure a test plan to use a test setting for all its manual tests, and a test setting for all its automated tests if you need it. In addition, you can select an environment that includes the set of roles in your test settings for all the manual tests, and you must select an environment for all the automated tests in your plan. If necessary, you can override these test settings and environments when you run your tests.
| Note |
|---|
If you run automated tests using Microsoft Test and Lab Manager, you must use an environment. |
You configure a test plan by using the test plan's Properties page in Microsoft Test and Lab Manager. Examples of test settings and environments for a test plan are shown in the following illustration.
.png)
Running Tests Using Microsoft Visual Studio
If you want to run your tests using Microsoft Visual Studio 2010, you can configure test settings to use when you run your tests. If you want to run your tests, or collect data, or affect a test machine remotely, you must specify a test agent controller to use in your test settings. The test agent controller will have agents that can be used for each role in your test settings. For more information about test agent controllers and test agents, see Setting Up Test Machines to Run Tests or Collect Data.
| Note |
|---|
Environments are only used when you run your tests using Microsoft Test and Lab Manager. |
Additional Information about Environments
You can create physical or virtual environments using Microsoft Test and Lab Manager. A physical environment might use physical computers or virtual machines. A virtual environment uses only virtual machines. An environment consists of a set of roles. A role specifies the purpose of one or more computers in the environment. For example, a specific role could be "Web Server". You can run tests, collect data, or affect a test machine for each specific role in your environment.
You must specify a test agent controller for your environment. Each computer with a test agent registered to that controller is called a machine and is available to use for your environment. For more information, see Setting Up Test Machines to Run Tests or Collect Data.
The following illustration shows how you define the set of roles for your test settings and then you can select an environment with machines assigned to each role to use when you run your tests. You can select any environment that includes at least the set of roles defined in your test settings. The environment may include other roles that are not specified in your test settings as shown in the following illustration.
.png)
Test Settings Configuration Details for Diagnostic Data Adapters
The following table provides an overview of the various ways that the diagnostic data adapters can be configured to be used with local or remote machines:
Diagnostic data adapter used in test setting | Manual Tests on local machine | Automated Tests | Manual Tests - collecting data by using a set of roles and an environment | Notes |
|---|
Action log and action recording: You can create a test setting that includes action recordings and action logs that will be saved with the test results for the test. You can play back the action recording later or you can view the action log to see what you did. | Yes | No | Yes (See Notes) | |
ASP.NET Client Proxy for IntelliTrace and Test Impact: This proxy allows you to collect information about the http calls from a client to a Web server for the IntelliTrace and Test Impact diagnostic data adapters. | Yes | Yes | Yes | |
ASP.NET profiler: You can create a test setting that includes ASP.NET profiling, which collects performance data on ASP.NET Web applications. | No | Yes (See Notes) | No | |
Code coverage: You can create a test setting that includes code coverage information that is used to investigate how much of your code is covered by tests. | No | Yes (See Notes) | No | You can only use code coverage when you run an automated test from Microsoft Visual Studio 2010 or mstest.exe, and only from the machine that is running the test. Remote collection is not supported. Collecting code coverage data does not work if you also have the test setting configured to collect IntelliTrace information.
|
IntelliTrace: You can configure the diagnostic data adapter for IntelliTrace to collect specific diagnostic trace information to help isolate bugs that are difficult to reproduce. This creates an IntelliTrace file that has an extension of .tdlog that collects this information. When you run your test and a test step fails, you can create a bug. The IntelliTrace file is automatically attached to this bug. The data that is collected in the IntelliTrace file increases debugging productivity by reducing the time that is required to reproduce and diagnose an error in the code. From this IntelliTrace file the local session can be recreated on another computer, this reduces the possibility of a bug being non-reproducible. For more information, see Debugging With IntelliTrace. | Yes | Yes | Yes | If you enable collecting IntelliTrace data, collecting code coverage data will not work. If you are using IntelliTrace for a Web client role, you must also select the ASP.NET Client Proxy for IntelliTrace and Test Impact diagnostic data adapter.
|
Event log: You can configure a test setting to include event log collecting, which will be included in the test results. | Yes | Yes | Yes | |
Network emulation: You can specify that you want to place an artificial network load on your test using a test setting. Network emulation affects the communication to and from the machine by emulating a particular network connection speed, such as dial-up. Note Network emulation cannot be used to increase the network connection speed. | Yes | Yes | Yes | |
System information: A test setting can be set up to include the system information about the machine that the test is run on. | Yes | Yes | Yes | |
Test impact: You can collect information about which methods of your applications code were used when a test case was running. This can be used together with changes to the application code made by developers to determine which tests were impacted by those development changes. | Yes | Yes | Yes | |
Video Recorder: You can create a video recording of your desktop session while you run a test. The video can help other team members isolate application issues that are difficult to reproduce. | Yes | Yes (See Notes) | Yes | |