Visual C++ 2005 Beta 1
Microsoft Visual C++ 2005
Microsoft Visual C++
Summary: Discusses fixing a problem with Invoking MSIL Functions under Loader Lock in Microsoft Visual C++ 2005. (8 printed pages)
In Microsoft Visual C++ 2002 and Microsoft Visual C++ 2003, any DLL compiled with the /CLR compiler option was at risk of nondeterministically deadlocking when loaded. This issue was termed the mixed DLL loading problem and was documented with partial workarounds in the following articles: the technical article Mixed DLL Loading Problem, and the Knowledge Base article PRB: Linker Warnings When You Build Managed Extensions for C++ DLL Projects.
The mixed DLL loading problem is centered around the Microsoft Windows OS loader lock. When a pure .NET assembly is loaded into a process, the /CLR loader can perform all of the necessary loading and initialization tasks itself. However, when that assembly contains any native code or data, the Windows loader must be used to process those constructs. The Windows loader guarantees that no code can access code or data in that DLL before it has been initialized, and that no code can redundantly load the DLL while it is partially initialized. To do this, the Windows loader uses a process global critical section that prevents these unsafe accesses from taking place. As a result, the loading process is vulnerable to many classic deadlock scenarios. These scenarios are manifested by the following two key issues.
First, if users attempt to execute functions compiled to Microsoft intermediate language (MSIL) when the loader lock is held (from DllMain or in static initializers, for example), this could cause deadlock. Consider the case in which the MSIL function references a type in an assembly that has not been loaded. The /CLR will attempt to automatically load that assembly, which may require the Windows loader to block on the loader lock. Since the loader lock is already held by code earlier in the call sequence, a deadlock results. However, executing MSIL under loader lock does not guarantee that a deadlock will occur, making this scenario difficult to diagnose and fix. In some circumstances, such as where the DLL of the referenced type contains no native constructs and all of its dependencies contain no native constructs, the loader lock is not actually required to load the .NET assembly of the referenced type. Additionally, the required assembly or its mixed native/.NET dependencies may have already been loaded into the AppDomain by other code. Consequently, the deadlocking could be difficult to predict, and could change behaviors based on the configuration of the target machine.
Second, when loading DLLs in versions 1.0 and 1.1 of the .NET Framework, the /CLR assumed that the loader lock was not held and performed several actions that are invalid under loader lock. While this is a valid assumption for pure .NET DLLs, mixed DLLs execute native initialization routines, which require the native Windows loader and therefore the loader lock to process. Consequently, even if the developer was not attempting to execute any MSIL functions during DLL initialization, there was still a small possibility of nondeterministic deadlock with versions 1.0 and 1.1 of the .NET Framework.
In Microsoft Visual C++ 2005, all nondeterminism has been removed from the mixed DLL loading process. The /CLR no longer makes false assumptions when loading mixed DLLs. Furthermore, the /CLR will refuse to JIT and run MSIL in debug scenarios when the loader lock is held. Additionally, the C++ compiler puts all .NET initialization in a separate initialization path outside of the loader lock. This enables developers to more easily perform mixed DLL initialization and immediately identify initialization problems when they do arise.
The Visual C++ 2005 Beta 1 compiler and runtime will generate a diagnostic when user code attempts to execute MSIL functions when the loader lock is held. The diagnostic window is pictured in Figure 1.
Figure 1. MSIL under Loader Lock diagnostic window
This document describes the remaining scenarios for which MSIL can execute under the loader lock, resolutions for the issue under each of those scenarios, debugging techniques with Visual C++ 2005 Beta 1 for determining the source of MSIL under loader lock issues, and improvements that are coming after Beta 1.
There are several different situations under which user code can execute MSIL under loader lock in Beta 1. The developer must ensure that the user code implementation is not MSIL under each of these circumstances. The following subsections describe all possibilities with a discussion of how to resolve issues in the most common cases.
The DllMain function is a user defined entry point for a DLL. Unless the user specifies otherwise, DllMain is invoked every time a process or thread attaches to or detaches from the containing DLL. Since this invocation can occur while the loader lock is held, no user-supplied DllMain function must ever be compiled to MSIL. Furthermore, no function in the call tree rooted at DllMain can be compiled to MSIL. To resolve issues here, the code block that defines DllMain should be modified with #pragma unmanaged, or the object file containing DllMain should be compiled without the /CLR switch. The same must be done for every function that DllMain calls. In cases where these functions must call a function that requires an MSIL implementation for other calling contexts, a duplication strategy can be used wherein both a .NET and a native version of the same function are created. This is a workable strategy because user-supplied DllMain functions should have rather simple implementations. The rules for code that can be executed in DllMain are very strict, which usually limits the complexity of the implementation of DllMain. For more information about DllMain rules, see this article. Alternatively, if DllMain is not required or if it does not need to be executed under loader lock, the user-provided DllMain implementation can be removed, which will eliminate the problem.
Static initializers are expressions that define the initial values for global static variables. These expressions, which may be function calls, constructor invocations, or C++ expressions that cannot be statically evaluated, are wrapped by a compiler generated initializer function which is called by the CRT. The code listing in Listing 1 shows five examples of static initializers: a simple expression, a function call, object construction, pointer initialization, and a static variable initialization. Note that this code would have to exist in a global scope for these examples to qualify as static initializers.
// Statically evaluatable – NO static initializer function generated double d = 0.0; // Static initializer function generated int a = init(); CFoo gFoo(arg1, arg2); CFoo *pgFoo = new CFoo(arg1, arg2);
Listing 1. C++ Static initalizer code examples
Static initializers defined in a file compiled with the /CLR option enabled will be invoked in a .NET initialization routine that is run by the CRT outside of loader lock. This routine is the .cctor method belonging to the loaded module, and is new to the Visual C++ 2005 release. The module .cctor is an MSIL function that is guaranteed by the /CLR to run before any other .NET code or data from the module can be accessed. Consequently, it is the appropriate vehicle for performing .NET initialization for DLLs.
Static initializers that are defined in a file compiled without the /CLR option will continue to be invoked during native initialization under the loader lock. This must be done for compatibility reasons, since the compiler has no specific knowledge when compiling any one source file that the resulting object file may be linked with object files compiled with /CLR. Since these static initializers execute under the loader lock, they are not permitted to call any functions that have MSIL implementations.
To solve the problem in the static initializer case there are three options: first, the source file containing the static initializer definition could be compiled with the /CLR option; second, any functions called by the static initializer can be compiled to native code using either the #pragma unmanaged directive, or by compiling the containing source file without the /CLR option; finally, it is possible to manually clone the code that the static initializer calls, providing both a .NET and a native version with different names. Developers would then call the native version from native static initializers and call the .NET version elsewhere.
Functions on Which Libraries Depend at Startup
There are several potentially user-supplied functions on which libraries depend for initialization during startup. When globally overloading operators in C++, the user-provided overloads are used everywhere. This includes STL initialization and destruction. The most common case is for operator new and delete overloads. STL and user-provided static initializers will invoke any user-provided versions of these operators. If the user-provided versions are compiled to MSIL, then these initializers will be attempting to call an MSIL implemented function while the loader lock is held. Additionally, a user-supplied malloc has the same consequences. To resolve this problem, any of these overloads or user-supplied definitions must be implemented as native code using the #pragma unmanaged directive.
If the user provides a custom global locale, this locale will be used for initializing all future I/O streams, including those that are statically initialized. If this global locale object was compiled to MSIL, then locale-object member functions compiled to MSIL may be invalidly called while the loader lock is held.
To solve the problem in the custom locale case, there are three options: first, the source files containing all global I/O stream definitions can be compiled using the /CLR option. This will prevent their static initializers from being executed under loader lock; second, the custom locale function definitions can be compiled to native code, either by using the #pragma unmanaged directive or by compiling the containg source files with the /CLR option disabled; finally, the developer could refrain from setting the custom locale as the global locale until after the loader lock is released. Then, the developer could imbue any already created I/O streams with this locale.
In some cases in Beta 1, it is difficult to detect the source of deadlocks, caused when trying to execute MSIL code while the loader lock is held. The following subsections discuss these scenarios and ways to work around these issues.
Implementation in Header
When function implementations exist in headers, all of the above issues can be complicated in Beta 1. In the case of both inline functions and template code, the implementations of functions must be specified in a header file. The C++ language specifies the One Definition Rule (ODR), which forces all implementations of functions with the same name to be semantically equivalent. Consequently, the C++ linker need not make any special considerations when merging object files that have duplicate implementations of a given function. The linker simply chooses the largest of these semantically equivalent definitions, to accommodate forward declarations and scenarios when different optimization options are used for different source files. This creates a problem for mixed native/.NET DLLs.
Since the same header may be included both by a CPP files with /CLR enabled and disabled, it is possible to have both MSIL and native versions of functions that provide implementations in headers. MSIL and native implementations have different semantics with respect to initialization under the loader lock, which effectively violates the one definition rule. Consequently, when the linker chooses the largest implementation, it may choose the MSIL version of a function, even if it was explicitly compiled to native code elsewhere using the #pragma unmanaged directive. To ensure that an MSIL version of a template or inline function is never called under loader lock, every definition of every such function called under loader lock must be modified with the #pragma unmanaged directive, or compiled in a source file with /CLR disabled. Often, the easiest way to achieve this is to push and pop the #pragma unmanaged directive around the #include directive for the offending header file. This strategy will not work for headers that contain other code that must directly call .NET APIs.
Diagnose in Debug Mode
All diagnoses of potential MSIL under loader lock violations should be done in Debug mode. Release mode may not produce diagnostics, and the optimizations performed in Release mode may mask some of the MSIL under loader lock scenarios. Specifically, Visual C++ 2005 performs an optimization called function-cloning, which will create both .NET and native implementations, rather than a .NET implementation and a native entry point forwarder thunk, for functions that meet certain criteria. One of these criteria is that the function must be smaller than a specific size. Obviously, a function's size can change due to simple edits, indirectly changing the cloning decisions made by the compiler. Under Debug mode, using the /Od compiler option, the function cloning optimization is disabled. To avoid situations where a small code change introduces a new observable case of the MSIL under loader lock problem, all diagnoses should be done in Debug mode.
The diagnostic that the /CLR generates in the Beta 1, when an MSIL function is invoked, causes the /CLR to suspend execution. This diagnostic therefore causes the Visual Studio 2005 Beta 1 mixed-mode debugger to hang when running the debuggee in-process. When attaching to the process, it is not possible to obtain a managed callstack for the debuggee using the mixed debugger. The ultimate goal is to identify the specific MSIL function that was called under loader lock. Given an MSIL under loader lock diagnostic in Beta 1, developers should complete the following steps to identify the offending function. Note that this process will be substantially improved after Beta 1.
- Ensure that symbols for mscoree.dll and mscorwks.dll are available.
This can be done in two ways. First, the PDBs for mscoree.dll and mscorwks.dll can be added to the symbol search path. To do this, open the symbol search path options dialog. (From the Tools menu, click Options. In the left pane of the Options dialog box, Open the Debugging node and click Symbols.) Add the path to the mscoree.dll and mscorwks.dll PDB files to the search list. These PDBs are installed with the .NET SDK to the %VSINSTALL%\SDK\v2.0\symbols. Click OK.
Second, the PDBs for mscoree.dll and mscorwks.dll can be downloaded from the Microsoft Symbol Server. To configure Symbol server in Beta 1, open the symbol search path options dialog. (From the Tools menu, click Options. In the left pane of the Options dialog box, Open the Debugging node and click Symbols.) Add the following search path to the search list: http://msdl.microsoft.com/download/symbols. Add a symbol cache directory to the symbol server cache text box. Click OK.
- Set debugger mode to native-only mode.
To do this, open the Properties grid for the startup project in the solution. Under the Configuration Properties subtree, select the Debugging Node. Set the Debugger Type Field to Native-Only.
- Start the Debugger (F5).
- When the /CLR diagnostic is generated, click Retry and then click Break.
- Open the call stack window. (From the Debug menu, click Windows, then Call Stack.) If the offending DllMain or static initializer is identified with a green arrow, as illustrated in Figure 2, skip to Step 10. If the offending function is not identified, the following steps must be taken to find it.
Figure 2. Sample call stack identifying an offending function
- Open the immediate window (From the Debug menu, click Windows, then Immediate.)
- Type .load sos.dll into the immediate window to load the SOS debugging service.
- Type !dumpstack into the immediate window to obtain a complete listing of the internal /CLR stack.
- Look for the first instance (closest to the bottom of the stack) of either _CorExeMain (if DllMain causes issue) or _VTableBootstrapThunkInitHelperStub (if static initializer causes issue). The stack entry just below this call is the invocation of the MSIL implemented function that attempted to execute under loader lock.
Figure 3 illustrates the Immediate Window with the !dumpstack output for an application with the static initializer version of the MSIL under loader lock problem. The line circled in red is the _VTableBootstrapThunkInitHelperStub invocation. The line circled in green is the offending static initializer for the global variable a.
Figure 3. Sample immediate window identifying an offending function
- Go to the source file and line number identified in Step 9 and correct the problem using the scenarios and solutions described in the Scenarios section.
After Beta 1, the /CLR diagnostic will be modified to provide better Visual Studio integration. Consequently, there will be no need to use the SOS debugging extensions DLL in the immediate window to identify the specific MSIL function that is being invoked while the loader lock is held.
Additionally, the Visual C++ team is investigating possible solutions to enable easier identification and resolution of MSIL under loader lock issues. When the Beta 2 tools ship, an updated version of this document, describing the process for eliminating MSIL under loader lock scenarios, will be published.
About the author
Scott Currie is a Program Manager on the Microsoft Visual C++ team.