Performing Exploratory Testing Using Microsoft Test Manager
Exploratory testing means testing an application without a set of tests defined in advance, and without a script of predetermined steps.
Microsoft Test Manager (MTM) helps you by recording the actions you perform as you work with your application. You can also record screenshots, comments, file attachments, audio narration and screen video. The recording makes it easier to trace any fault that you might discover. You can also store your actions as a test case, so that it is easy for you or a colleague to replicate the test after the application is updated.
Visual Studio Ultimate, Visual Studio Premium, Visual Studio Test Professional
In Microsoft Test Manager, open Testing Center, Test, Do Exploratory Testing.
- or –
Select a requirement work item, and then choose Explore work item. This associates the recording of your test with the work item.
The Exploratory Testing window opens, and waits until you are ready to start.
Prepare to run your application. For example, if your application is a website, start the web server.
In the Exploratory Testing window, choose Start.
Run the application and explore its features. For example, open a web browser and log in to your website.
The Exploratory Testing window records the actions you perform in the rest of the screen. You can add comments, screenshots and files as you work. They will be added to any bug or test case that you create.
[Visual Studio 2012 Update 1] The action log automatically includes a snapshot of the screen, focusing on the area around the text or gesture. The screenshots are included when you create a bug in the exploratory session.
If you are exploring a particular requirement, verify that the requirement is satisfied under a variety of different conditions. For more information, see What exploratory tests should I perform?
When you find flaws in the application, choose Create bug.
To help reproduce the error, the steps you performed will automatically be saved in the bug, in the Steps to Reproduce pane. Comments that you wrote during the test also appear, along with the attachments and screenshots that you added, and additional system information.
When the bug is created, can change which steps you want to be included.
You can generate a test case at the same time as the bug, containing the same steps. This helps to ensure that the bug does not recur in the future. The bug and the test case are linked.
[Visual Studio 2012 Update 1] The description of each action is automatically accompanied by a screenshot of the area near the action.
To see how the whole screen appeared during the test, choose Action Log.
When the application is updated or developed further, or when a bug is fixed, you will want to re-run your tests to make sure everything is still working - or to see if it works better.
But there's a substantial amount of expertise, creative thinking, and experimentation in an exploratory test. To save time on future occasions, you can save your actions as a script of steps in a test case. When it is time to perform these tests again, you - or someone else - only have to follow the steps, instead of re-inventing them.
You can create a test case either directly from your exploratory session, or immediately after you create a bug.
You can adjust the number of recent steps that are included in the test case.
If you create a test case directly from an exploratory session, you will typically spend some time practicing with a feature before performing a sequence of steps that you want to record. Edit the test case to start where your sequence begins.
You should also edit the work item to state what result should be seen after each step.
When you save and close the work item, you can return to exploration.
Create separate test cases for each separate aspect of the requirement.
When a fix for the bug has been checked in, open the MTM and choose Testing Center, Test, Verify Bugs. This page has a list of bugs that were created in this test plan and that are linked to test cases. Select the bug that has been fixed and choose Verify. The Test Runner will open and show the steps that you performed to find the error. Follow the steps and verify that the error does not occur. Mark the test as passed and close the bug.
If your application is a website or a client-server application, you can collect information from the server machines, as well as from your own client machine.
To do this, you have to set up a lab environment and install your servers on the machines in that environment. For more information, see Running Tests on a Lab Environment.
You must also configure your tests to collect data from the environment. You can either do this in the test plan properties, or you can choose this option in individual tests.
To configure an individual test session, start the test by using Explore with Options.
To configure all tests to collect server data: In MTM, choose Testing Center, Plan, Properties. At Test Environment, choose the environment on which you have installed your server.
You can set the properties of the test plan so that additional data are recorded in your test session and in any bugs that you create. For example, you can add or remove programs from which user actions should be collected.
You can also capture screen video as you work, and audio commentary.
For more information, see How to: Choose Test Settings and Environments for a Test Plan.