TN_1214: Gathering code coverage information from the command line

Ian Huff, Software Design Engineer
Microsoft Corporation

Included in Visual Studio Team System is a useful new code coverage tool. Code coverage can tell you exactly what percent of your code is being exercised by your test methods. We have taken the time to integrate this functionality with the testing framework included in Visual Studio Team System and this is the recommended way to collect code coverage information, but the IDE and the testing framework is not the only way to collect code coverage information. We have also provided command line tools and APIs that will allow you to integrate code coverage information collection into your own custom test harnesses or build events. This TechNote will cover how to collect and view code coverage information without having to go through the Visual Studio Team System IDE.

The first main step to collecting code coverage information is to instrument the assemblies that you are interested it. Instrumentation will insert code into the assemblies so that when they are run, code coverage information will be collected. You can instrument both managed and native dll files, exe files and assemblies. The tool that you use to instrument is called VsInstr.exe and it is found in the following directory:

Microsoft Visual Studio 8\Team Tools\Performance Tools

Instrumenting the assembly is done by passing the –coverage option to the VsInstr.exe tool:

vsinstr –coverage MyAssembly.exe

When you instrument the assembly you are modifying it permanently, so the VsInstr tool automatically makes a backup of the file you instrument. In this case the backup will be called MyAssembly.orig.exe.

The next step in collecting code coverage information is to make sure that the collection monitor is running so that our collected coverage data gets written somewhere. The coverage monitor is called VsPerfMon.exe and is located in the same location as VsInstr.exe was located. To start it up for code coverage collection just use the following command:

start vsperfmon –coverage –output:mytestrun.coverage

Now the command shell will be running your monitor, waiting for some data to collect and write to the output file. Now is the time where you run your test suites or exercise your instrumented code in whatever manner you want to collect coverage information for. The command window will wait until the instrumented code has exited before it will close down and create the coverage file. Once it has, you will now have a code coverage file called mytestrun.coverage that details what code you exercised in your instrumented assembly while the monitor was running. You can open this file directly from Visual Studio to see a code coverage results window that will give you a breakdown of what percent of the code in your instrumented assembly was covered.

The above sample shows the basics of collecting command line code coverage information, but we didn’t stop with just providing command line tools. We also provide assemblies that you can use to build code coverage functionality into your own projects.

First off, we provide a controller dll that performs all the instrumentation and collection functions that we detailed above. This dll is called:

Microsoft.VisualStudio.Coverage.Monitor.dll

Below is some simple example code that uses some of the functions provided by the above dll to collect code coverage information:

using System;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics;
using Microsoft.VisualStudio.CodeCoverage;
 
// You must add a reference to Microsoft.VisualStudio.Coverage.Monitor.dll
 
namespace CoverControl
{
       class CoverProgram
       {
              static void Main(string[] args)
              {
                     // Write something to call vsinstr.exe
                     Process p = new Process();
                     StringBuilder sb = new StringBuilder(
"/COVERAGE ");
                     sb.Append("myassembly.exe");
                     p.StartInfo.FileName = "vsinstr.exe";
                     p.StartInfo.Arguments = sb.ToString();
                     p.Start();
                     p.WaitForExit();
                     // TODO: Look at return code – 0 for success
 
                     // A guid is used to keep track of the run
                     Guid myrunguid = Guid.NewGuid();
                     Monitor m = new Monitor();
                     m.StartRunCoverage(myrunguid,
"mytestrun.coverage");
 
                     // TODO: Launch some tests or something
                     // that can exercise myassembly.exe
 
                     // Complete the run
                     m.FinishRunCoverage(myrunguid);
              }
       }
}

The code above will do essentially the same thing as we did from the command line. First we instrument the assembly, then we start up the monitor, exercise the code (you have to fill in this part) and finally shut down the monitor. Like from the command line, we will end up with a mytestrun.coverage file at the end of the program execution.

The other assembly that we provide for code coverage is:

Microsoft.VisualStudio.Coverage.Analysis.dll

This assembly provides the smarts to extract the data from a .coverage file and display it in some useful fashion. Below is an example that uses this assembly to dump out the coverage information as XML.

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.VisualStudio.CodeCoverage;
 
// You must add a reference to Microsoft.VisualStudio.Coverage.Analysis.dll
 
namespace CoverDump
{
       class DumpProgram
       {
              static void Main(string[] args)
              {
                     // Create a coverage info object from the file
                     CoverageInfo ci = CoverageInfoManager.CreateInfoFromFile("myfile.coverage");
 
                     // Ask for the DataSet
             // The parameter must be null
                     CoverageDS data = ci.BuildDataSet(null);
 
                     // Write to XML
                     data.Lines.WriteXml("mylines.xml");
              }
       }
}

After running this you will generate a mylines.xml file that contains the coverage information. Instead of outputting as XML you could also perform your own visualizations with the DataSet at this point.

So as you can see, not only did we provide solid integration of code coverage with the testing framework, but we also provided all the tools and APIs needed to work with code coverage on your own terms. I would like to suggest that the best way to start with code coverage is to work thought the unit testing framework, but hopefully this TechNote will help you if you want to dig in deeper.