TN_1206: Profiling ASP.NET in instrumentation mode from the command line

Ian Huff, Software Design Engineer
Microsoft Corporation

Included in Visual Studio Team System 2005 for Developers (and Suite edition) is the new performance profiler tool. This tool allows you to diagnose performance issues using both sampling and instrumentation in native and managed executables and dll files. We also support directly profiling ASP.NET applications from the IDE; you can treat them just like any other native or managed application. While we spend a lot of time to get ASP.NET support working correctly from the IDE, we realize that not everyone will always be able to use the IDE for all of their scenarios. For instance, the Visual Studio IDE does eat up some space while you are profiling that may affect performance and we do force a restart of IIS before we start gathering ASP.NET data. This TechNote will take a look at using some of the command line tools to use instrumentation to collect data from ASP.NET applications outside of the IDE.

The first step in profiling is to choose if you are using sampling or instrumentation profiling. In a nutshell, sampling takes a snapshot of program state in periodic cycles while instrumentation inserts code into your assembly to catch every function entry and exit. This causes instrumentation to collect very large amounts of data so we usually recommend that users start with sampling profiling to isolate performance hotspots. Then they can use instrumentation to dig in deeper into more specific profiling scenarios. However, for ASP.NET profiling, we actually recommend working mainly with instrumentation mode. This is due to the fact that in sampling mode lots of data will show up from the ASP.NET framework and obscure what is happening in your specific code. This TechNote focuses just on instrumentation mode.

ASP.NET is very complex and your code is usually only a very small part of the overall system behavior. From the perspective of the profiler there are two categories of user code for ASP.NET. First, there are pre-built assemblies that are usually placed in the bin directory of the website. And second, there are dynamically built assemblies that are from generated from code embedded in a webpage or in the App_Code directory. The following section will show you how to gather data for these two types of code in instrumentation modes. All of these samples assume that you have Visual Studio Team System for Developers or Team Suite installed on the machine, but you won’t need to open up the IDE for anything. The various tools that we are using (vsperfclrenv, vsperfcmd, vsinstr and vsperfmon) are all found in the Microsoft Visual Studio 8\Team Tools\Performance Tools directory.

Instrumented Profiling of Pre-built Assemblies:

These are assemblies that are placed in the bin directory of the web site and are not dynamically built by ASP.NET.

• vsperfclrenv /globaltraceon
  • This will set up the environment variables that tell .NET to load our profiling helpers.
• reboot (as of this writing reboot is still required)
  
  
• vsinstr [my assembly path]
  • vsinstr needs to be run on any pre-built assemblies that you wish to gather performance data on.
• vsperfmon /TRACE /OUTPUT:[your ouputfile].vsp /USER:"[ASP.NET worker process user]"
  • This will start the performance monitor that will collect the data as your application runs and write it out to the output file you specify.
    
    
  • See Appendix A to find the user name that the ASP.NET worker process uses (it depends on what version of IIS you have running).
    
    
  • vsperfmon will run throughout the collection process, so you must start another command prompt to complete other commands.
• Run your test scenario against the web application
  
  
• iisreset /STOP
  • The profiled process must be shut down for collection to complete.
• vsperfcmd -SHUTDOWN
  
  
• vsperfreport [your outputfile].vsp /SUMMARY:FUNCTION /PACKSYMBOLS
  • PACKSYMBOLS is necessary to get the symbols pinned correctly before the target assemblies are lost from their location in temporary storage; it also preserves symbols so the report can be meaningful if opened on another machine for analysis.
• iisreset /START
  • Only do this if you are going to do more profiling.
• vsperfclrenv /globaloff
  • This step and the following step will prevent our profiling helpers from being loaded into ASP.NET in the future. This should be done when you have completed profiling, as loading our helpers can affect performance.
• reboot
  
  
• Open [your outputfile].vsp in Visual Studio.
  • Since you packed the symbols in a previous step, you can copy [your outputfile].vsp to any machine that has a full install of VSTS and open in there.

Instrumented Profiling of Dynamically Built Assemblies

These are all of the assemblies that contain code from your web pages, code behind files and App_Code directories.

• vsperfclrenv /globaltraceon
  
  
• reboot
  
  
• Back up the web.config file
  
  
• Fix up the web.config file for your site
  • See Appendix B for instructions on how to fix up the web.config file.
• vsperfmon /TRACE /OUTPUT:[your ouputfile].vsp /USER:"[ASP.NET worker process user]"
  
  
• Run your test scenario against the web application.
  
  
• iisreset /STOP
  • The profiled process must be shut down for collection to complete
• vsperfcmd -SHUTDOWN
  
  
• vsperfreport [your outputfile].vsp /SUMMARY:FUNCTION /PACKSYMBOLS
  
  
• Restore the web.config file (if you don't intend to do more profiling)
  
  
• iisreset /START
  
  
• vsperfclrenv /globaloff (if you don't intend to do more profiling)
  
  
• reboot
  
  
• Open [your outputfile].vsp in Visual Studio.

You can profile both pre-built and dynamically generated assemblies at the same time by following the instructions for dynamically built assemblies but by also manually running vsinstr against the pre-built assemblies prior to fixing the web.config file.

Appendix A: Finding the User Name and Process Id for the ASP.NET Worker Process

• Make certain that your ASP.NET application is running
  
  
• Open the task manager
  
  
• Go to the Processes Tab
  
  
• Show the PID and User Name columns
  • Do this by going to the view menu and choosing select columns and select PID and User Name
• If aspnet_wp.exe exists use it, otherwise look for w3wp.exe.
  
  
• On win2K3Srv there may be multiple worker processes depending on how isolation is set up. In this case, use %windir% \system32\iisapp.vbs to correlate w3wp PIDs with IIS application pools.
  
  
• Read the PID and User Name Columns (defaults listed below)
  • Windows Server 2003/IIS 6: (w3wp.exe, NETWORK SERVICE)
    
    
  • Windows XP/IIS 5.1: (aspnet_wp.exe, ASPNET)
    
    
  • Windows 2000/IIS 5.0: (aspnet_wp.exe, LOCAL SYSTEM)

Appendix B: Modifying the web.config File

1. Add or modify the compilation tag:

<system.web> <compilation assemblyPostProcessorType="Microsoft.VisualStudio.Enterprise.Common.AspPerformanceInstrumenter, Microsoft.VisualStudio.Enterprise.ASPNetHelper, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" /> </system.web>

Note that the PublicKeyToken must match the PublicKeyToken of the ASPNetHelper.dll.

2. Set up the runtime tag to show the location of the ASPNetHelper.dll file

<runtime> <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1"> <dependentAssembly> <assemblyIdentity name="Microsoft.VisualStudio.Enterprise.ASPNetHelper" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" /> <codeBase version="8.0.0.0" href="file:///C:/Program%20Files/Microsoft%20Visual%20Studio%208/Common7/IDE/PrivateAssemblies/Microsoft.VisualStudio.Enterprise.ASPNetHelper.DLL" /> </dependentAssembly> </assemblyBinding> </runtime>

Note that the PublicKeyToken must match the PublicKeyToken of the ASPNetHelper.dll. Also, the HREF in codebase must be a file URL (not just a pathname) pointing to this .DLL.

3. Set up the location of vsinstr.exe and its dependencies

<appSettings> <add key="Microsoft.VisualStudio.Enterprise.AspNetHelper.VsInstrLocation" value="C:\Program Files\Microsoft Visual Studio 8\Team Tools\Performance Tools\vsinstr.exe" /> <add key="Microsoft.VisualStudio.Enterprise.AspNetHelper.VsInstrTools" value="C:\Program Files\Microsoft Visual Studio 8\Team Tools\Performance Tools\" /> </appSettings>

• VsInstrLocation is the full path including the filename of vsinstr.exe.
  
  
• VsInstrTools is the directory containing helper dlls for vsinstr.exe that aren’t available elsewhere on the path, these are msdis150.dll and mspdb80.dll.

Appendix C: Troubleshooting

Symptom: Everything seemed to work, but when I tried to open the file I got an error that ‘File contains no data’

There are two likely candidates for this.

1. The /USER switch was not correctly set when starting vsperfmon.exe. Check and make certain that you correctly identified the process identity of the ASP.NET worker process (see appendix B) and make certain that you provide it to vsperfmon.exe.
   
   
2. No assemblies were instrumented or the instrumented assemblies were never run. The command prompt that hosts vsperfmon.exe will show a message when it starts getting data from an instrumented process. If you never see that message, something went wrong. Make certain that the web.config file was correctly modified. If you manually instrument an assembly, make certain it is the one that is being run.

Symptom: My web page shows an error that vsperf80.dll can’t be found

This is because vsperf80.dll must be on the path for the ASP.NET worker process. Both the Visual Studio setup and the stand-alone performance tools setup should place vsperf80.dll in [Windows]\system32. To fix this just locate vsperf80.dll and place it in your [Windows]\system32 directory.

Symptom: Everything seemed to work, but I'm not seeing all the symbols that I expect.

There are two likely candidates for this.

1. If you aren't seeing any symbols in your own code, make sure you run vsperfreport with the /PACKSYMBOLS flag BEFORE you restart the server.
   
   
2. If you aren't seeing any symbols in ASP.NET, make sure you have the symbol server set up correctly

    Set your _NT_SYMBOL_PATH variable to “symsrv*symsrv.dll*c:\localcache*http://msdl.microsoft.com/download/symbols” 

Need more help with symbols? Check out http://www.microsoft.com/whdc/devtools/debugging/symbolpkg.mspx.