You can run test cases with associated automation from the command line using tcm.exe instead of using the user interface provided by Microsoft Test Manager. This enables you to start runs automatically from a batch file. For example, you can then decide to schedule that batch file to run every time that a build is completed.
To run these automated test cases so that the test results are part of your test plan, you must use a physical or virtual environment. When you start the run from the command line, use tcm.exe. You must specify both the test settings that contain the set of roles that you require to run your tests, and an environment that contains this set of roles. For more information about test settings and environments, see Setting Up Test Machines to Run Tests or Collect Data.
You can use a query to specify the test case and configuration pairing, which are named test points, or you can specify the test suites and configurations that you want to run from your test plan. You have to know the IDs for your test plan, your test suites, and configurations. You can use tcm.exe to list these for you if it is required.
Use the following procedures to gather the information that you have to have in order to run your automated tests from the command line and then run the tests:
List All the Configurations in Your Team Project
To list all the configurations in your team project
Open a Visual Studio command prompt.
To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt.
The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC
To list all the configurations in your team project, type the following at the command prompt:
Tcm configs /list /server:<path for your Team Foundation Server> /teamproject:<Team Project name>
.gif) Note |
|---|
The path for your Team Foundation Server typically has the following format: http://<name of your Team Foundation Server>:8080/tfs. If you do not know the correct path, contact your Team Foundation system administrator.
|
You will see results similar to these:
Id Name
---------------- ------------------
1 Vista and IE 7
25 Vista and IE 8
| Note |
|---|
You can list plans for your team project too by using the following command: Tcm plans /list /server:<path for your Team Foundation Server> /teamproject:<Team Project name> |
List All the Test Plans in Your Team Project
To list all the test plans in your team project
Open a Visual Studio command prompt.
To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt.
The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC
To list all the plans in your team project, type the following at the command prompt:
Tcm plans /list /server:<path for your Team Foundation Server> /teamproject:<Team Project name>
| Note |
|---|
The path for your Team Foundation Server typically has the following format: http://<name of your Team Foundation Server>:8080/tfs. If you do not know the correct path, contact your Team Foundation system administrator.
|
You will see results similar to these:
Id Name
---------------- ------------------
1 Beta 1 Test Plan
22 RTM Test Plan
(Optional) To find all the test suites in a test plan, type the following at the command prompt:
Tcm suites /list /planid:<your plan id> /server:<path for your Team Foundation Server> /teamproject:<Team Project name>
You will see results similar to these:
Id Name
---------------- ------------------
1 Priority 1 Test Suite
15 Shopping Cart Test Suite
List All the Test Environments for Your Team Project
To list all the test environments for your team project
Open a Visual Studio command prompt.
To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt.
The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC
To list all the plans in your team project, type the following at the command prompt:
Tcm testenvironments /list /server:<path for your Team Foundation Server> /teamproject:<Team Project name>
| Note |
|---|
The path for your Team Foundation Server typically has the following format: http://<name of your Team Foundation Server>:8080/tfs. If you do not know the correct path, contact your Team Foundation system administrator.
|
You will see results similar to these:
Name Description
---------------- ------------------
Staging Environment Use to test staging environment
Testing Environment for Customer ABC
List All the Test Suites in Your Test Plan
To list all the test suites in your test plan
Open a Visual Studio command prompt.
To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt.
The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC
To find all the test suites in your test plan, type the following at the command prompt:
Tcm suites /list /planid:<your plan id> /server:<path for your Team Foundation Server> /teamproject:<Team Project name>
You will see results similar to these:
Id Name
---------------- ------------------
1 Priority 1 Test Suite
15 Shopping Cart Test Suite
Create a Test Run Using a Test Point Query
To create a test run using a test point query
Open a Visual Studio command prompt.
To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt.
The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC
To create a test run using a test point query for a specific test configuration using the information in the following table.
Type the following at the command prompt:
Tcm run /create /title:"Nightly Run" /planid:1 /querytext:"SELECT * FROM TestPoint WHERE ConfigurationId=13" /settingsname:"Automated Settings" /testenvironment:remote /server:<path for your Team Foundation Server> /teamproject:<Team Project name>
You will see results similar to these:
Run created with ID: <xxx>.
To view the progress of your test run, follow the steps in View and Update the Test Results.
Create a Test Run For a Test Suite and a Specific Configuration
To create a test run for a test suite and a specific configuration
Open a Visual Studio command prompt.
To do this, click Start, point to All Programs, point to Microsoft Visual Studio 2010, point to Visual Studio Tools, and then click Visual Studio 10.0 Command Prompt.
The command prompt opens to the folder: <drive>:\Program Files\Microsoft Visual Studio 10.0\VC
To create a test run for all the tests in a test suite for a specific configuration using the information in the following table.
Id | Value |
|---|
Planid | 1 |
Suiteid | 5 |
Configid | 13 |
Type the following at the command prompt:
Tcm run /create /title:"Nightly Run" /planid:1 /suiteid:5 /configid:13 /settingsname:"Automated Settings" /testenvironment:remote /server:<path for your Team Foundation Server> /teamproject:<Team Project name>
You will see results similar to these:
Run created with ID: <xxx>.
To view the progress of your test run, follow the steps in View and Update the Test Results.
View and Update the Test Results
To view and update the test results
Open Microsoft Test Manager.
| Note |
|---|
To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager. |
To view the test results, click the down-arrow on the center group switcher and click Testing Center.
On the center group menu bar, click Test and then click Analyze Test Runs.
The Analyze Test Runs activity is displayed that shows any test runs for this test plan.
Double-click the test run with the title that you gave it from the command line to open and view the details.
The test run details are displayed.
(Optional) To update the title of your test run, type the new name in Title.
(Optional) If your test failed, you can update the reason for the failure. Click Resolution and select the reason for the failure from the list.
(Optional) To add comments to the test result, click the Comments icon. Type your comments and then click Save comments.
(Optional) To view the details of an individual test, double-click the test.
The test result is displayed that shows the details from the test run and the test results history for that test. You can close this view to return to the test run.
| Note |
|---|
If you determine that there is a bug from your analysis, you can create a bug from this view. |
To save these changes for this test run, click Save in the toolbar.
Concepts