What's New for Application Lifecycle Management Testing in Visual Studio 2012
Visual Studio 2012.2 includes new features in the following areas:
Test plan cloning now available in Microsoft Test Manager
You can now clone a test plan in Microsoft Test Manager by selecting the Test Plan Manager in the Organize tab of the Testing Center. Performing cloning operations on test plans is still supported using tcm.exe from the command line. See Copying and Cloning Test Suites and Test Cases.
Testing SharePoint 2013 apps
Visual Studio 2012.1 includes new features in the following areas:
Screenshots are automatically added to the action log when you create a bug work item during an exploratory test session. See Performing Exploratory Testing Using Microsoft Test Manager.
Use link queries to add requirements to the test plan. For example, find all the requirements that don’t have tests. See Organizing Test Cases Using Test Suites.
Testing SharePoint 2010 apps
Easier configuration for third-party unit test frameworks in team build definitions for Team Build
Publish test results to Team Foundation Server from the command line
You can publish test results to Team Foundation Server from the command line by using the /logger:TfsPublisher option for VSTest.Console.exe. See Using VSTest.Console from the Command Line.
Visual Studio 2012.1 with Team Foundation Server Update 1 includes new features in the following areas:
Copy requirements along with test suites from one team project to another. See Copying and Cloning Test Suites and Test Cases.
Pause a test run, save it, and later resume it after doing other work or even running other tests. See How to: Run Manual Tests.
Test suites based on link queries – for example, define a suite of all tests linked to requirements in the Active state. See Organizing Test Cases Using Test Suites.
Visual Studio 2012 includes new features in the following areas:
Test project compatibility with Visual Studio 2010
Test projects that you created in Visual Studio 2010 SP1 that contain unit tests, coded UI tests, Web performance or load tests can be opened in Visual Studio 2012. Therefore, your team can continue to use Visual Studio 2010 SP1 or Visual Studio 2012 to work with these test projects. In some cases, like when projects contain coded UI tests, the projects might be repaired when they are opened . The repair process enables the test project to work correctly with both Visual Studio 2010 SP1 and Visual Studio 2012. Additionally, Visual Studio 2012 includes some changes for test settings (.tesetsettings files) that were introduced in Visual Studio 2010. These changes are minor for Microsoft Test Manager and for web performance and load test types in Visual Studio Ultimate. However, for unit test and coded UI test types, the changes are more significant. For more information, see:
New test project templates
Visual Studio 2012 introduces new test project templates. In Visual Studio 2010, there was a single test project template. Visual Studio 2012 has these test project templates:
Unit Test Project – Can include unit tests, third-party unit tests, ordered tests, and generic tests.
Coded UI Test Project – Can include the test types listed for Unit Test Project above, and coded UI tests.
Web Performance and Load Test Project – Can include Web performance tests and load tests.
For more information, see Upgrading Tests from Earlier Versions of Visual Studio.
Upgrading Lab Management
If you are using lab environments, the version of your test controller, test agents, Team Foundation Server, and workflow definitions must be the same. Lab environments that you created with Microsoft Test Manager 2010 on Team Foundation Server 2010 can be opened with Microsoft Test Manager on Visual Studio Team Foundation Server 2012 after you upgrade your Lab Management components. When you upgrade your Lab Management components, you must perform specific steps during the upgrade process or your lab environments won't work.
If you upgrade to Visual Studio Team Foundation Server 2012 and Visual Studio 2012, you do not have to upgrade SCVMM 2008 R2 to SCVMM 2012 for you lab environments to work. Both versions of SCVMM are compatible with Lab Management on Visual Studio 2012. See Upgrading Lab Management Components from Visual Studio 2010.
Microsoft Test Manager
Coded UI tests
Web performance and load tests
Microsoft Test Manager 2010 with Visual Studio Team Foundation Server 2012:
You can connect Visual Studio Team Foundation Server 2012 from Microsoft Test Manager 2010 and use most of the features. However, to connect to this version of Team Foundation Server, you must first install Visual Studio 2010 Service Pack 1 and all the latest software updates on the machine running Microsoft Test Manager 2010.
You can’t create or edit lab environments. You can, however, view the ones that have already been created, and use them for running manual or automated tests, and for remote data collection.
You can’t use any of the newer features of Visual Studio Team Foundation Server 2012 such as Exploratory Testing. To use these features, you must also upgrade to Microsoft Test Manager.
Microsoft Test Manager with Team Foundation Server 2010:
To use Microsoft Test Manager with Team Foundation Server 2010, you must first ensure that you are using Team Foundation Server 2010 SP1, and then you must install the latest software updates for the Team Foundation Server. Similarly, all the test controllers and test agents installed in lab environments must be running version 2010 SP1 components, and you must install the latest software updates on all of them.
The Environments and Library tabs in Microsoft Test Manager are disabled when you connect to Team Foundation Server 2010. For this functionality, you must install Microsoft Test Manager 2010 SP1 side-by-side with Microsoft Test Manager.
You can’t use any of the newer features of Microsoft Test Manager such as Exploratory Testing. To use these features, you must also upgrade the server to Visual Studio Team Foundation Server 2012.
You can use Microsoft Test Manager to run manual tests to help you identify problems in your Windows Store apps that are running on a remote device, such as a tablet.
The action log file associated with testing Windows Store apps or Internet Explorer 10 sessions on Windows 8 are saved as .html files containing screenshots for each action step conducted during the test.
This .html file can be viewed in your Web browser. The enhanced action log makes reproducing bugs easier since you can see the exact steps taken by the tester. These enhanced action logs are also displayed when you submit a bug from either Microsoft Test Runner, or the Exploratory Test window.
See Testing Windows Store apps Running on a Device Using Microsoft Test Runner, Testing Windows Store apps Running on a Device Using the Exploratory Test Window and How to: View Manual Test Case Results Using Microsoft Test Manager.
You can conduct exploratory testing from Microsoft Test Manager. Exploratory testing is the testing of an application without a set of tests defined in advance. It can also be described as simultaneous test planning and test execution. During an exploratory test session, you are not restricted to a script or a set of predetermined steps like you are with a manual test.
For more information about manual tests see Creating Manual Test Cases Using Microsoft Test Manager.
Exploratory test sessions can be associated with requirements work item types, such as user stories. You can start an exploratory test session from Microsoft Test Manager and conduct appropriate testing on the application using the Exploratory Testing window to verify that the product backlog item behaves correctly. This testing is unscripted and doesn’t have predefined test cases. This can be useful if you are using a design and test approach while developing your application, because the Exploratory Testing window provides an easy way to create manual test cases based on the action steps during your session. If you create a manual test based on the action steps that are recorded during your exploratory session, the manual test case is automatically associated with your product backlog item and placed in the appropriate test suite. All bugs that you file during this session will also be automatically linked with the product backlog item. For more information about creating user stories in Microsoft Test Manager, see How to: Add Product Backlog Items, User Story, or Requirements Work Items to Your Test Plan and How to: Create a Work Item using Microsoft Test Manager.
From Microsoft Test Manager, you can also conduct general exploratory testing that is not associated with a product backlog item.
While you explore your application using the Exploratory Testing window, you can create bugs that include the action steps performed and any comments, screenshots, and file attachments you have added.
Test steps can include multiple lines to consolidate related actions within a single test step in your test cases.
Microsoft Test Manager includes a toolbar that you can use to format the text of your test steps. You can use various formatting options, for example bold, underline, or color highlighting to emphasis key points.
The Plan tab in the center group menu bar of Microsoft Test Manager includes Results which provides a report on the status of your test plan.
You can clone test suites into other test plans. Cloning lets you work on different releases in parallel. For example, if you already have a test plan called “Contoso V1 – Milestone 1” and your team decides to make version V2, you can create a clone of the test plan to be used for the V2 source code branch. When you clone a test plan, the new destination test plan will have the same area path as the original source test plan, but will have a different iteration path. After cloning the test plans, you and your team can work on both versions of the applications simultaneously.
The user experience for both manual testing and the new exploratory testing on a machine that has ARM-based architecture is identical to that on other platforms. However, when using a test machine with ARM architecture, you cannot include action recordings. Only the following diagnostic data adapters are supported:
Including coded UI tests in a SharePoint app lets you verify that the whole app, including its UI controls, is functioning correctly. Coded UI tests can also validate values and logic in the user interface. See Testing SharePoint 2010 Applications with Coded UI Tests and Predefined Extraction Rules for SharePoint Apps.
The Coded UI Test Editor is included in Visual Studio 2012. The Coded UI Test Editor was available to MSDN subscribers as a feature pack in Visual Studio 2010. See Microsoft Visual Studio 2010 Feature Pack 2.
The Coded UI Test Editor lets you easily modify your coded UI tests. Using the Coded UI Test Editor, you can locate, view, and edit your test methods, UI actions and their associated controls in the UI control map. See Editing Coded UI Tests Using the Coded UI Test Editor.
Test projects that were created in Visual Studio 2010 are silently repaired when they are opened in Visual Studio 2012. If the test projects are checked into source control, the project files are checked out for this repair. Once repaired, these test projects containing coded UI tests will work in both Visual Studio 2010 SP1 and Visual Studio 2012.
There is a known issue regarding references in coded UI test projects not appearing in Solution Explorer. For more information, see the ReadMe file included on the Visual Studio 2012 installation media.
For more information, see Upgrading Coded UI Tests from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.
Coded UI test logs filter and record important information about coded UI tests, allowing for debugging issues quickly. The logs are presented in HTML format using your web browser.
For more information, see Analyzing Coded UI Tests Using Coded UI Test Logs.
Coded UI tests support record, playback, and validation of HTML5 controls including Audio, Video, ProgressBar and Slider.
Internet Explorer 10 includes spell checking capabilities for all text boxes, allowing you to choose from a list of suggested corrections. Coded UI Test will ignore user actions such as selecting an alternative spelling suggestion. Only the final text typed into the text box will be recorded.
The following actions are recorded for coded UI tests that use the spell checking control: Add to Dictionary, Copy, Select All, Add To Dictionary, and Ignore.
In Internet Explorer 9, pinned sites were introduced. With pinned sites, you can get to your favorite sites directly from the Windows taskbar—without having to open Internet Explorer first. Coded UI tests can now generate intent aware actions on pinned sites. See Pinned Sites.
Internet Explorer 9 introduced the following semantic tags: section, nav, article, aside, hgroup, header, footer, figure, figcaption and mark. Coded UI tests ignore all of these semantic tags while recording. You can add assertions on these tags using the Coded UI Test Builder. You can use the navigation dial in the Coded UI Test Builder to navigate to any of these elements and view their properties.
There are differences in between the way Internet Explorer 8, Internet Explorer 9, and Internet Explorer 10 handle white space characters. Coded UI Test handles these differences seamlessly. Therefore, a coded UI test created in Internet Explorer 8 for example, will play back successfully in Internet Explorer 9, and Internet Explorer 10.
The coded UI test APIs include new functionality. Some of this new functionality includes:
Find column names and column values for List View controls.
Get content of controls like Data Grid, or List and validate it.
Get tool tip text of control.
Find broken links on the page.
For more information, see API Reference for Testing Tools for Visual Studio ALM.
You can use web performance and load tests on your SharePoint 2010 apps to verify performance and stress abilities. Load tests can be configured to emulate conditions such as user loads, browser types, and network types. See Web Performance and Load Testing SharePoint 2010 and 2013 Applications.
With [Visual Studio 2012.2] installed, you can also use web performance and load tests on your SharePoint 2013 apps. See Web Performance and Load Testing SharePoint 2010 and 2013 Applications.
Your existing Web performance and load tests that were created in Visual Studio 2010 will continue to run in Visual Studio 2012. However, Visual Studio 2012 has deprecated the Test View window that you might have used previously to run your Web performance and load tests. To run your Web performance and load tests, you must now do so from the Web Performance Test Editor or the Load Test Editor window. See Running Load and Web Performance Tests.
If you are using test controllers from Visual Studio for Web performance or load testing, then the version of test controller must match the version of Visual Studio. For more information about upgrading test controllers, see Upgrading Test Controllers from Visual Studio 2010 and Installing and Configuring Test Agents and Test Controllers.
For more information about upgrading Web performance and load tests, see Upgrading Web Performance and Load Tests from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.
Lab agents and test agents from previous versions of Lab Management have been combined in to a single agent—a test agent. When you use Microsoft Test Manager to run tests, you no longer have to install test agents on the computers that you compose into a lab environment. Lab Management does this for you when you create or repair your lab environments.
For more information, see Creating an SCVMM Environment Using Stored Virtual Machines and Templates and Creating a Standard Environment.
Lab Management for Visual Studio 2012 supports SCVMM 2012 and clustered Hyper-V hosts. For more information, see Creating an SCVMM Environment Using Stored Virtual Machines and Templates. For information about upgrading SCVMM 2008 R2 to SCVMM 2012, see Upgrading SCVMM 2008 R2 to SCVMM 2012.
You do not have to upgrade SCVMM 2008 R2 to SCVMM 2012. Lab Management on Visual Studio 2012 supports both versions of SCVMM.