What's New for Application Lifecycle Management Testing in Visual Studio 2012

  • Article

Visual Studio 2012.2 includes new features in the following areas:

  • Test plan cloning now available in Microsoft Test Manager

    You can now clone a test plan in Microsoft Test Manager by selecting the Test Plan Manager in the Organize tab of the Testing Center. Performing cloning operations on test plans is still supported using tcm.exe from the command line. See Copying and Cloning Test Suites and Test Cases.

  • Testing SharePoint 2013 apps

    • Web performance and load tests for SharePoint 2010 and SharePoint 2013 apps

Visual Studio 2012.1 includes new features in the following areas:

Visual Studio 2012.1 with Team Foundation Server Update 1 includes new features in the following areas:

Visual Studio 2012 includes new features in the following areas:

Test project compatibility with Visual Studio 2010

Test projects that you created in Visual Studio 2010 SP1 that contain unit tests, coded UI tests, Web performance or load tests can be opened in Visual Studio 2012. Therefore, your team can continue to use Visual Studio 2010 SP1 or Visual Studio 2012 to work with these test projects. In some cases, like when projects contain coded UI tests, the projects might be repaired when they are opened . The repair process enables the test project to work correctly with both Visual Studio 2010 SP1 and Visual Studio 2012. Additionally, Visual Studio 2012 includes some changes for test settings (.tesetsettings files) that were introduced in Visual Studio 2010. These changes are minor for Microsoft Test Manager and for web performance and load test types in Visual Studio Ultimate. However, for unit test and coded UI test types, the changes are more significant. For more information, see:

New test project templates

Visual Studio 2012 introduces new test project templates. In Visual Studio 2010, there was a single test project template. Visual Studio 2012 has these test project templates:

  • Unit Test Project – Can include unit tests, third-party unit tests, ordered tests, and generic tests.

  • Coded UI Test Project – Can include the test types listed for Unit Test Project above, and coded UI tests.

  • Web Performance and Load Test Project – Can include Web performance tests and load tests.

For more information, see Upgrading Tests from Earlier Versions of Visual Studio.

Upgrading Lab Management

If you are using lab environments, the version of your test controller, test agents, Team Foundation Server, and workflow definitions must be the same. Lab environments that you created with Microsoft Test Manager 2010 on Team Foundation Server 2010 can be opened with Microsoft Test Manager on Visual Studio Team Foundation Server 2012 after you upgrade your Lab Management components. When you upgrade your Lab Management components, you must perform specific steps during the upgrade process or your lab environments won't work.

If you upgrade to Visual Studio Team Foundation Server 2012 and Visual Studio 2012, you do not have to upgrade SCVMM 2008 R2 to SCVMM 2012 for you lab environments to work. Both versions of SCVMM are compatible with Lab Management on Visual Studio 2012. See Upgrading Lab Management Components from Visual Studio 2010.

Microsoft Test Manager

  • Compatibility of Microsoft Test Manager with Visual Studio 2010

  • Manual Testing Windows Store Apps

  • Enhanced Action Logs for Windows Store Apps

  • Exploratory Testing window

  • Manual Test Steps Can Include Multiple Lines

  • Manual Tests Includes Rich Text

  • Microsoft Test Manager Test Plan Results

  • Cloning Test Suites into Other Plans for New Iterations

Unit tests

Coded UI tests

  • Compatibility of Coded UI test projects with Visual Studio 2010

  • Coded UI Test Editor

  • Coded UI Test Logs

  • Support for HTML5 in Internet Explorer 9 and Internet Explorer 10

  • Support for Internet Explorer 10 Spell Checking

  • Support for 64-bit Internet Explorer with Windows 8

  • Support for Pinned Sites in Internet Explorer 9

  • Support for Internet Explorer 9 Semantic Tags

  • Seamless Handling of White Space Characters between Versions of Internet Explorer

  • The Notification Area of Internet Explorer Are Now Recorded With the “Continue on Error” Attribute Set

  • New Coded UI Test APIs

Web performance and load tests

  • Removal of the Virtual User Limit for Load Testing

  • Compatibility of Web performance and load tests with Visual Studio 2010

Lab Manager

  • Simplified Agent Configuration

  • SCVMM 2012 and Clustered Host Support

Microsoft Test Manager

Compatibility of Microsoft Test Manager with Visual Studio 2010

Microsoft Test Manager 2010 with Visual Studio Team Foundation Server 2012:

  • You can connect Visual Studio Team Foundation Server 2012 from Microsoft Test Manager 2010 and use most of the features. However, to connect to this version of Team Foundation Server, you must first install Visual Studio 2010 Service Pack 1 and all the latest software updates on the machine running Microsoft Test Manager 2010.

  • You can’t create or edit lab environments. You can, however, view the ones that have already been created, and use them for running manual or automated tests, and for remote data collection.

  • You can’t use any of the newer features of Visual Studio Team Foundation Server 2012 such as Exploratory Testing. To use these features, you must also upgrade to Microsoft Test Manager.

Microsoft Test Manager with Team Foundation Server 2010:

  • To use Microsoft Test Manager with Team Foundation Server 2010, you must first ensure that you are using Team Foundation Server 2010 SP1, and then you must install the latest software updates for the Team Foundation Server. Similarly, all the test controllers and test agents installed in lab environments must be running version 2010 SP1 components, and you must install the latest software updates on all of them.

  • The Environments and Library tabs in Microsoft Test Manager are disabled when you connect to Team Foundation Server 2010. For this functionality, you must install Microsoft Test Manager 2010 SP1 side-by-side with Microsoft Test Manager.

  • You can’t use any of the newer features of Microsoft Test Manager such as Exploratory Testing. To use these features, you must also upgrade the server to Visual Studio Team Foundation Server 2012.

See Upgrading Microsoft Test Manager from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.

Manual testing of Windows Store apps

You can use Microsoft Test Manager to run manual tests to help you identify problems in your Windows Store apps that are running on a remote device, such as a tablet.

See Testing Windows Store apps Running On a Device with Microsoft Test Manager.

Enhanced action logs for Windows Store apps

The action log file associated with testing Windows Store apps or Internet Explorer 10 sessions on Windows 8 are saved as .html files containing screenshots for each action step conducted during the test.

Enhanced action log

This .html file can be viewed in your Web browser. The enhanced action log makes reproducing bugs easier since you can see the exact steps taken by the tester. These enhanced action logs are also displayed when you submit a bug from either Microsoft Test Runner, or the Exploratory Test window.

See Testing Windows Store apps Running on a Device Using Microsoft Test Runner, Testing Windows Store apps Running on a Device Using the Exploratory Test Window and How to: View Manual Test Case Results Using Microsoft Test Manager.

Exploratory testing window

You can conduct exploratory testing from Microsoft Test Manager. Exploratory testing is the testing of an application without a set of tests defined in advance. It can also be described as simultaneous test planning and test execution. During an exploratory test session, you are not restricted to a script or a set of predetermined steps like you are with a manual test.

Tip

For more information about manual tests see Creating Manual Test Cases Using Microsoft Test Manager.

Exploratory test sessions can be associated with requirements work item types, such as user stories. You can start an exploratory test session from Microsoft Test Manager and conduct appropriate testing on the application using the Exploratory Testing window to verify that the product backlog item behaves correctly. This testing is unscripted and doesn’t have predefined test cases. This can be useful if you are using a design and test approach while developing your application, because the Exploratory Testing window provides an easy way to create manual test cases based on the action steps during your session. If you create a manual test based on the action steps that are recorded during your exploratory session, the manual test case is automatically associated with your product backlog item and placed in the appropriate test suite. All bugs that you file during this session will also be automatically linked with the product backlog item. For more information about creating user stories in Microsoft Test Manager, see How to: Add Product Backlog Items, User Story, or Requirements Work Items to Your Test Plan and How to: Create a Work Item using Microsoft Test Manager.

From Microsoft Test Manager, you can also conduct general exploratory testing that is not associated with a product backlog item.

Exploratory Tester

While you explore your application using the Exploratory Testing window, you can create bugs that include the action steps performed and any comments, screenshots, and file attachments you have added.

See Performing Exploratory Testing Using Microsoft Test Manager.

Manual test steps can include multiple lines

Multiline test case

Test steps can include multiple lines to consolidate related actions within a single test step in your test cases.

see Add action and validation test steps to your manual test case.

Manual tests includes rich text

Microsoft Test Manager includes a toolbar that you can use to format the text of your test steps. You can use various formatting options, for example bold, underline, or color highlighting to emphasis key points.

See Add action and validation test steps to your manual test case.

Microsoft Test Manager test plan results

The Plan tab in the center group menu bar of Microsoft Test Manager includes Results which provides a report on the status of your test plan.

See How to: View Test Plan Results in Microsoft Test Manager.

Test plan results

Cloning test suites into other plans for new iterations

You can clone test suites into other test plans. Cloning lets you work on different releases in parallel. For example, if you already have a test plan called “Contoso V1 – Milestone 1” and your team decides to make version V2, you can create a clone of the test plan to be used for the V2 source code branch. When you clone a test plan, the new destination test plan will have the same area path as the original source test plan, but will have a different iteration path. After cloning the test plans, you and your team can work on both versions of the applications simultaneously.

Cloning test suites

See Copying and Cloning Test Suites and Test Cases.

Partial support for ARM architecture

The user experience for both manual testing and the new exploratory testing on a machine that has ARM-based architecture is identical to that on other platforms. However, when using a test machine with ARM architecture, you cannot include action recordings. Only the following diagnostic data adapters are supported:

See Specifying Test Settings in Microsoft Test Manager and Create Test Settings for Automated System Tests Using Microsoft Test Manager.

Coded UI tests

Coded UI tests for SharePoint 2010 apps

[Visual Studio 2012.1]

Including coded UI tests in a SharePoint app lets you verify that the whole app, including its UI controls, is functioning correctly. Coded UI tests can also validate values and logic in the user interface. See Testing SharePoint 2010 Applications with Coded UI Tests and Predefined Extraction Rules for SharePoint Apps.

Coded UI Test Editor

The Coded UI Test Editor is included in Visual Studio 2012. The Coded UI Test Editor was available to MSDN subscribers as a feature pack in Visual Studio 2010. See Microsoft Visual Studio 2010 Feature Pack 2.

The Coded UI Test Editor lets you easily modify your coded UI tests. Using the Coded UI Test Editor, you can locate, view, and edit your test methods, UI actions and their associated controls in the UI control map. See Editing Coded UI Tests Using the Coded UI Test Editor.

Compatibility of coded UI test projects with Visual Studio 2010

Test projects that were created in Visual Studio 2010 are silently repaired when they are opened in Visual Studio 2012. If the test projects are checked into source control, the project files are checked out for this repair. Once repaired, these test projects containing coded UI tests will work in both Visual Studio 2010 SP1 and Visual Studio 2012.

Warning

There is a known issue regarding references in coded UI test projects not appearing in Solution Explorer. For more information, see the ReadMe file included on the Visual Studio 2012 installation media.

For more information, see Upgrading Coded UI Tests from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.

Coded UI test logs

Coded UI test logs filter and record important information about coded UI tests, allowing for debugging issues quickly. The logs are presented in HTML format using your web browser.

For more information, see Analyzing Coded UI Tests Using Coded UI Test Logs.

Support for HTML5 in Internet Explorer 9 and Internet Explorer 10

Coded UI tests support record, playback, and validation of HTML5 controls including Audio, Video, ProgressBar and Slider.

See Using HTML5 Controls in Coded UI Tests and Supported Configurations and Platforms for Coded UI Tests and Action Recordings.

Support for Internet Explorer 10 spell checking

Internet Explorer 10 includes spell checking capabilities for all text boxes, allowing you to choose from a list of suggested corrections. Coded UI Test will ignore user actions such as selecting an alternative spelling suggestion. Only the final text typed into the text box will be recorded.

The following actions are recorded for coded UI tests that use the spell checking control: Add to Dictionary, Copy, Select All, Add To Dictionary, and Ignore.

Support for 64-bit Internet Explorer with Windows 8

Previously 64-bit versions of Internet Explorer were not supported for recording and playback. With Visual Studio 2012, coded UI tests have been enabled for 64-bit versions of Internet Explorer running under Windows 8.

Support for pinned sites in Internet Explorer 9

In Internet Explorer 9, pinned sites were introduced. With pinned sites, you can get to your favorite sites directly from the Windows taskbar—without having to open Internet Explorer first. Coded UI tests can now generate intent aware actions on pinned sites. See Pinned Sites.

Support for Internet Explorer 9 semantic tags

Internet Explorer 9 introduced the following semantic tags: section, nav, article, aside, hgroup, header, footer, figure, figcaption and mark. Coded UI tests ignore all of these semantic tags while recording. You can add assertions on these tags using the Coded UI Test Builder. You can use the navigation dial in the Coded UI Test Builder to navigate to any of these elements and view their properties.

Seamless handling of white space characters between versions of Internet Explorer

There are differences in between the way Internet Explorer 8, Internet Explorer 9, and Internet Explorer 10 handle white space characters. Coded UI Test handles these differences seamlessly. Therefore, a coded UI test created in Internet Explorer 8 for example, will play back successfully in Internet Explorer 9, and Internet Explorer 10.

The notification area of Internet Explorer is Now Recorded With the “Continue on Error” Attribute Set

All actions on the notification area of Internet Explorer are now recorded with the “Continue on Error” attribute set. If the notification bar doesn’t appear during playback, the actions on it will be ignored and coded UI test will continue with the next action.

New coded UI test APIs

The coded UI test APIs include new functionality. Some of this new functionality includes:

  • Find column names and column values for List View controls.

  • Get content of controls like Data Grid, or List and validate it.

  • Get tool tip text of control.

  • Find broken links on the page.

For more information, see API Reference for Testing Tools for Visual Studio ALM.

Web Performance and Load Tests

Web performance and load tests for SharePoint 2010 and SharePoint 2013 apps

[Visual Studio 2012.1]

You can use web performance and load tests on your SharePoint 2010 apps to verify performance and stress abilities. Load tests can be configured to emulate conditions such as user loads, browser types, and network types. See Web Performance and Load Testing SharePoint 2010 and 2013 Applications.

With [Visual Studio 2012.2] installed, you can also use web performance and load tests on your SharePoint 2013 apps. See Web Performance and Load Testing SharePoint 2010 and 2013 Applications.

Removal of the Virtual User Limit for Load Testing

The limit on the number of virtual users that you can use in your load tests has been removed. You no longer have to purchase virtual user licenses to increase the number of virtual users that you can use in your load tests.

Compatibility of Web performance and load tests with Visual Studio 2010

Your existing Web performance and load tests that were created in Visual Studio 2010 will continue to run in Visual Studio 2012. However, Visual Studio 2012 has deprecated the Test View window that you might have used previously to run your Web performance and load tests. To run your Web performance and load tests, you must now do so from the Web Performance Test Editor or the Load Test Editor window. See Running Load and Web Performance Tests.

If you are using test controllers from Visual Studio for Web performance or load testing, then the version of test controller must match the version of Visual Studio. For more information about upgrading test controllers, see Upgrading Test Controllers from Visual Studio 2010 and Installing and Configuring Test Agents and Test Controllers.

For more information about upgrading Web performance and load tests, see Upgrading Web Performance and Load Tests from Visual Studio 2010 and Upgrading Tests from Earlier Versions of Visual Studio.

Lab Manager

Simplified agent configuration

Lab agents and test agents from previous versions of Lab Management have been combined in to a single agent—a test agent. When you use Microsoft Test Manager to run tests, you no longer have to install test agents on the computers that you compose into a lab environment. Lab Management does this for you when you create or repair your lab environments.

For more information, see Creating an SCVMM Environment Using Stored Virtual Machines and Templates and Creating a Standard Environment.

SCVMM 2012 and clustered host support

Lab Management for Visual Studio 2012 supports SCVMM 2012 and clustered Hyper-V hosts. For more information, see Creating an SCVMM Environment Using Stored Virtual Machines and Templates. For information about upgrading SCVMM 2008 R2 to SCVMM 2012, see Upgrading SCVMM 2008 R2 to SCVMM 2012.

Note

You do not have to upgrade SCVMM 2008 R2 to SCVMM 2012. Lab Management on Visual Studio 2012 supports both versions of SCVMM.

See Also

Reference

What's New in Developer Quality and Diagnostic Tools in Visual Studio 2012

Concepts

What's New in Visual Studio 2012

Testing the Application

Upgrading Tests from Earlier Versions of Visual Studio