Setting Up Machines and Collecting Diagnostic Information Using Test Settings
You can use Test settings in Microsoft Test Manager and Visual Studio to collect extra data when you run your tests. For example, you might want to make a video recording as you run your test. There are diagnostic data adapters to:
-
Collect each UI action step in text format
-
Record each UI action for playing back
-
Collect system information
-
Collect event log data
-
Collect IntelliTrace data to help isolate non-reproducible bugs
Diagnostic data adapters can also be used to change the behavior of a test machine. For example, with a test setting in Visual Studio, you can emulate various network topology bottlenecks to evaluate the performance of your team’s application.
In this topic:
With Microsoft Test Manager, you configure a test plan to run your tests. A test plan can have two test settings:
-
Manual runs
-
Automated runs
You create these test settings using the Properties page of the test plan in Microsoft Test Manager.
You can configure both of these test settings to use a lab environment which can emulate a single machine, or multiple machine roles. The test setting includes separate configuration settings for the types of data to collect for each machine role using diagnostic data adapters.
Use the following topics to help you create and configure test settings in Microsoft Test Manager.
|
Tasks |
Associated Topics |
|---|---|
|
Create test settings for use in Microsoft Test Manager: You can create separate test settings for manual and automated tests in a test plan. |
|
|
Update test settings in Microsoft Test Manager: You can make corrections to existing test settings by editing them in the Lab Center or from the Properties activity for a test plan. |
|
|
Choose existing test settings for a test plan in Microsoft Test Manager: Test settings are stored in Team Foundation Server and can be selected for use in multiple test plans. |
A lab environment is a collection of virtual or physical machines that you can use to develop and test applications. A lab environment can contain multiple machine roles needed to test multi-tiered applications, such as workstations, web servers, and database servers. You can create and manage lab environments and run manual tests in a lab environment using Microsoft Test Manager, or run automated tests in a lab environment. When you run your tests using a lab environment, the test will collect data, or affect the behavior of the machine for each specific machine role that you configured in your test settings. In addition, you can use a build-deploy-test workflow with your lab environment to automate the process of building, deploying, and running automated tests on your application.
The following illustration shows examples of test settings and environments for a test plan.
The following illustration shows how you define the set of machine roles for your test settings. You can then select a lab environment that has computers or virtual machines that are assigned to each machine role to use when you run your tests. You can select any lab environment that includes at least the set of machine roles that are defined in your test settings. The lab environment may include other machine roles that are not specified in your test settings, as shown in the following illustration.
To run your unit, coded UI, web performance, or load tests by using Visual Studio, you can add, configure and select the test settings to use when you run your tests. To run your tests, collect data, or affect a test machine remotely, you must specify a test controller to use in your test settings. The test controller will have agents that can be used for each role in your test settings.
Use the following topics to help you create and configure test settings for use with Visual Studio.
|
Tasks |
Associated Topics |
|---|---|
|
Create test settings to use in Visual Studio: You can create a test setting for your unit, coded UI, web performance and load tests using the supported diagnostic data adapters. |
|
|
Edit test settings in Visual Studio: You can make corrections to existing test settings by editing them. |
|
|
Specify test setting to use in your solution: You solution can contain multiple test settings files. You can select the test setting file to use, depending on your testing needs from among them. |
|
|
Configure your Visual Studio tests to use remote machines: To run your tests, collect data, or affect a test machine remotely, you can install and configure test controller and test agent. |
The following table provides an overview of the various ways that the diagnostic data adapters can be configured for use with local or remote machine roles.
|
Diagnostic data adapter that is used in test setting |
Manual Tests on local machine |
Automated Tests |
Manual Tests: Collecting data by using a set of roles and an environment |
Notes |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
Yes |
No |
Yes (See Notes) |
|
||||||||||
|
|
Yes |
Yes |
Yes |
|
||||||||||
|
|
No |
Yes (See Notes) |
No |
|
||||||||||
|
|
No |
Yes (See Notes) |
No |
|
||||||||||
|
|
Yes |
Yes |
Yes |
|
||||||||||
|
|
Yes |
Yes |
Yes |
For more information, see Debug your app by recording code execution with IntelliTrace. |
||||||||||
|
|
No |
Yes (See Notes) |
No |
You can use the network emulation diagnostic data adapter for a client or server role. You do not have to use the adapter on both these roles that communicate with each other.
|
||||||||||
|
|
Yes |
Yes |
Yes |
|
||||||||||
|
|
Yes |
Yes |
Yes |
|
||||||||||
|
|
Yes |
Yes (See Notes) |
Yes |
To do this, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop. |
Creating a Diagnostic Data Adapter to Collect Custom Data or Affect a Test Machine
You can create and use a custom diagnostic data adapter to collect custom data, or perform tasks to impact a machine such as reduce available system memory, reduce available disk space, or allocate and hold other resources during a test.
Running Manual Tests using Team Web Access
Using Microsoft Test Manager you can run manual tests, exploratory test sessions, and automated tests from a test plan. When you run any of these tests from your test plan, the diagnostic data collected will be included in the test results.
Running System Tests Using Microsoft Visual Studio
After configuring your test settings in Visual Studio, you can run your tests and collect diagnostic data to help isolate bottlenecks and bugs in your application.
For additional guidance, see
Note