How to: Analyze Manual Test Runs Using Microsoft Test Manager

If you used Microsoft Test Manager to run automated tests or if you ran them from the command line using tcm.exe, you can analyze your automated test runs to determine why tests have failed or examine any other issues associated with the test run.

As you analyze the test results for each test, you can select a failure type and resolution for each failed test, or add a comment about your analysis to this point. To help with your analysis, you can open the test result for each test to see all previous runs of that test.

If you determine that there is an issue, you can create a bug and associate the data from the test result for this test with the bug. If you determine that the failed test is caused by a known issue, you can link that test to an existing bug. When this bug is fixed, the tester who verifies the bug can then see all the test cases that have to be rerun to verify that the bug is fixed.

If you want to add data to the test run, you can add attachments or comments to the test run itself.

To help with analysis, you might want to run your automated test using the manual test steps in the associated test case. For more information, see How to: Run a Test Case with Associated Automation as a Manual Test.

The following illustration shows how to select the run that you want to analyze and view the details.

To analyze your test runs using Microsoft Test Manager

1. Open Microsoft Test Manager.

   Note

   To display the Microsoft Test Manager window, choose Start, and then choose All Programs. Point to Microsoft Visual Studio 2012 and then choose Microsoft Test Manager.

2. To view the test results, choose the down-arrow on the center group switcher and then choose Testing Center.

3. On the center group menu bar, choose Test and then choose Analyze Test Runs.

   The Analyze Test Runs activity is displayed. It shows any automated test runs for this test plan.

4. (Optional) To display manual test runs, choose Show manual runs.

   The manual test runs are also displayed in the list of test runs.

5. Double-click a test run to open it and view the details.

   The test run details are displayed.

6. (Optional) To update the title of your test run to be more meaningful, type the new name in Title.

7. (Optional) To add comments to the test run, type your comments in Comments.

8. For any failed tests, you can update the reason for the failure. Select the test in the list of tests, choose the drop-down arrow next to Failure type and select the reason for the failure from the list.

9. (Optional) For any failed tests, you can update the resolution of the failure. Select the test in the list of tests, choose the drop-down arrow next to Resolution and select the resolution from the list.

10. To create a bug if this failed test is caused by a new issue, choose Create bug in the toolbar.

    The New Bug activity is displayed. All the test results data is automatically added to the bug. The bug is associated with the test case, so that when the bug is fixed the tester knows which test case to rerun. Type the necessary data in the bug fields and choose Save and Close. For more information about how to add data to the bug, see How to: Submit a Bug Using Microsoft Test Manager.

11. To link this test to an existing bug, follow these steps:

    1. Select the test and choose Link to bug.

       The Link Test Result to Existing Bug dialog box is displayed.

    2. Edit the query as required.

    3. Choose Run to display the bugs from the query.

    4. Select a bug from the list, and then choose Link bug.

12. To add notes for a test result in the test run, follow these steps:

    1. Select the test and choose the Notes icon.

       The Notes for test result dialog box is displayed.

    2. Type in the notes and choose Save notes.

13. (Optional) To view the details of the test results for an individual test, double-click the test.

    The test result is displayed that shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run. For more information about this view, see How to: View Manual Test Case Results Using Microsoft Test Manager.

    Note

    After you examine the details and previous result history, you might determine that there is an issue. From this view, you can then create a bug that is associated with this test.

14. (Optional) To add an attachment to the test run, follow these steps:

    1. Choose the drop-down arrow next to Attachments

    2. Choose Add.

       The Add Attachment dialog box is displayed.

    3. Choose Browse to locate the attachment.

    4. Type a description of the attachment in Comment.

    5. Choose OK to add the attachment to the test run.

15. To save any changes for the test run, choose Save in the toolbar.

See Also

Tasks

How to: Run Automated Tests In a Lab Environment Using Microsoft Test Manager

Concepts

How to: Run Automated Tests from the Command Line Using Tcm

Running Automated Tests