CLR Run-Time Breaking Changes

In RTM, we had behavior on some of our IO classes which was simply inappropriate. One such case is DirectoryInfo.Parent. In RTM, if you called this API, and passed a path ending in a slash, then the path returned would simply give the same path, without the slash:

DirectoryInfo di = new DirectoryInfo(@"c:\temp\Dir\");
DirectoryInfo di2 = di.Parent();

In v1.0, the call to Parent would return "c:\temp\Dir" for the DirectoryInfo. However, this is not the way users expect this to behave: the backslash should simply have been ignored. We therefore changed this API to ignore a trailing backslash, so in v1.1, the above call will now return "c:\temp". As part of this change, DirectoryInfo.Name was changed. In v1.0, if a path which ended in a backslash, we would do no interpretation of the path at all, and simply return the fullname. In V1.1, we now determine that if you have specified 'c:\foo\bar', then the name simply refers to bar. This change in behavior is consistent with the change to Parent.

In the CLR model, we assert that arguments, locals, and return values (things which you can't tell their size) can be expanded at will. When you say float, it means anything greater than or equal to a float. So we can sometimes give you what you asked for 'or better'. When we do this, we can spill 'extra' data, almost like a 'you only asked for 15 precision points, but congratulations! We gave you 18!'. If someone expected the floating point precision to always remain the exactly the same, they could be affected. In order to faciliate performance improvements and better scenarios, the CLR may rewrite (as in this case) parts of the register. For example, things that used to truncate because of spilling, no longer do. We make these kinds of changes all the time. We believe this is an appropriate change, and it is even called out specifically in the CLI specification, as something which can, and will occur with different iterations:

Affected code will look something like the following:TypeSpec

#4 (1b000004)
-------------------------------------------------------
TypeSpec : Class System.Byte

It should however, look like:

TypeSpec #4 (1b000004)
-------------------------------------------------------
TypeSpec : ValueClass System.Byte

The CLR accepted this incorrect metadata in v1.1. We have tightened down and do not accept it in v2.0.

1. Make base classes that depend on COM clients COM visible. (preferred)
2. Change derived class to use class interface of type None and explicitly forward calls to base class methods. (preferred)
3. We would also add a config switch that would revert back to the old behavior. Note that wouldn't be guarantee that things will work either but there is a chance that they will if there were no changes in the base classes. If some changes are indeed made to the base class, clients could be broken by it as described above.

GetTypeFromCLSID() used to return a type object from tlbimp'ed assembly if there is a type with same GUID already loaded earlier. Such type used to yield 'false' when Type.IsComObject() is called. And, Type.GetMethodInfo() used to return whatever we could see in IL metadata assembly.

Now:

• GetTypeFromCLSID() always returns a RuntimeType regardless of whether a type is already loaded with same GUID.
• Type.IsComObject() returns true.
• And, Type.GetMethodInfo() doesn't fetch any of the methods defined in tlb.

The reason we need to do this is to make things deterministic. In the V1 and V1.1 CLR, if the type has been loaded then we will wrap the created instance with the actual type, even if the Interop assembly for that type hasn't been registered. This can lead to unfortunate race conditions where, depending on what has currently been run in the AppDomain, the type will either be wrapped with an __ComObject or with the actual type. To remove these race conditions, we changed the behavior to only wrap the created COM object with the actual type if the Interop assembly is registered.

The problem with the V1 and V1.1 behavior is that depending on the app and the environment in which it is run (is it ngen'd, is it run off a network share, what are the security settings of the machine, etc), the CLR might need to start up COM to perform some of the startup code. When this happens, the CLR needs to start up COM before control enters the user's main and because we need to pick a default, we would CoInitialize it to MTA. What this would cause is that in some situations, your code below that does a CoInitialize in your Main method would fail. However under other circumstances it would pass. Changing the CLR to never use COM before the user's code is executed wasn't possible (it's not impossible, it is just way too much work and puts too many restrictions on the CLR to be viable) so we decided to make the behavior deterministic by always CoInitializing to MTA unless the [STAThreadAttribute] was specified on the main method.

Input from Christopher Brumme on this change:

"Whenever you set the apartment state of a running managed thread, you are in a race condition. That's because things like .cctors or security demands or assembly load notifications may have executed on that thread, perhaps causing marshaling of oleaut data—or any number of other things that could have caused a CoInitialize to happen. Even if the first line of your managed threadproc sets the apartment state, we have already executed an arbitrary amount of managed application code. And that arbitrary set of code changes whenever you upgrade the CLR, change the JIT inlining, fiddle with .cctor rules for NGEN vs. domain neutral, change security policy, etc.

If you are trying to set it to MTA, chances are you are okay. That's because we will default to MTA if we need to CoInitialize a thread. The only way someone would have set it to STA is if a .cctor or security infrastructure or hosting code had done this to you. And that would be pretty rude. Of course, if the CLR sets it to MTA and then the app sets it to MTA, this will be quietly accepted. So the only case that's likely to break is if the application selects STA but the CLR has already CoInitialized the running thread to MTA.

Anyway, the bottom line is that applications which set the apartment state of a running managed thread are perched on a knife edge. Long term, we need to move to a more stable and predictable world. The only question is the best way to do this. >We needed to change the behavior of 'main' on a managed EXE to be MTA if it is unspecified and we added a config setting for old C++ apps and for VB/C# apps that weren't built with Visual Studio.

The other change we added is for managed threads started with 't = new Thread(...); t.Start();'. If you didn't set the apartment state before you called Start, then it is now too late. We will have already placed you in the MTA. The same config setting to get the old behavior applies here too.

We realize that both of these changes will break apps. But those apps are going to break at some rate, as we disturb the race conditions anyway. We are providing the deterministic and reliable behavior now, know it will break some applications, rather than leaving in non-deterministic behavior that breaks applications with each new release."

1. Anybody who was using Thread.CurrentThread.ApartmentState = ApartmentState.STA; as the first line in their Main method. For example, all WinForms applications generated with v1.1 C++ designer will break because of the such call in WinMain() that gets generated automatically. Note that they might be broken anyway (even without this change since they are dependent on any random changes in CLR initialization and on config changes).
2. Anybody who was not initializing new thread as STA before they started it (but used to do it as first line of code that executes on that thread, for example).

For Unicode standard compliance, Encoding.GetBytes() will not emit bytes if there is an unpaired or out of order surrogate. This is most obvious if the caller call GetBytes() with one high or low surrogate. In this case, UTF8Encoding and UnicodeEncoding will not emit nothing.

The Unicode 4.0 requires that compliant applications not emit unpaired surrogates. In v1.1, GetBytes() will emit bytes for lone surrogates if the encoding supports it (such as UTF-8 and UnicodeEndcoding). However, this leads CLR not to be Standard compliance with Unicode 4.0.

The change can break application's assumption about that GetBytes() will emit leading high surrogates or mismatched surrogates. BinaryWriter.Write(char ch) is one example of being broken.

Although breaking, we would need to go head and fix this to provide more consistent behavior, justification:

1. With the current behavior, Equals and GetHashCode are not implemented the same way, if two objects are compared equal they should have the same hash code. The current implementation of GetHashCode is based on the name of the sortkey and so returns random results.
2. The current behavior (based on instance? yuck!) has no true utility, where as the fixed behavior (creating a hashcode based on the string) does, since it could be used as an optimization for comparisons with a hashcode index and a sortkey index.
3. There is no other good way to give the behavior since we cannot add a CultureInfo or CompareInfo to a sortkey (which already has one implied by its parent object from which it was created).

Cctor ordering for beforefieldinit types is not specified by ECMA. ECMA only specifies that the runtime will execute cctors on beforefieldinit types some time prior to any access to a type's static field. It does not offer any guarantees regarding exactly when and in which order these cctors should be executed.

Note: C# automatically marks types as beforefieldinit when static fields are initialized using "=" syntax like in "static int i = 5;". VB.NET programs are not affected since they use precise cctor semantics always.

In v1.0 and v1.1, some pathways through the loader recorded a failure to load an assembly and would fail subsequent attempts to load that same assembly into the same AppDomain. However, most pathways through the loader would not cache this binding failure. This lack of caching enables certain scenarios, which some customers are doubtless taking advantage of.

In v2.0, the default behavior is to break these scenarios. Naturally the .NET Framework supports an opt-in mechanism to force the loader back to the v1.0 & v2.0 behavior via a config file. These scenarios are broken by the following changes made in v2.0:

• All binding failures per assembly per AppDomain are cached. Subsequent attempts would unconditionally continue to fail. Transient errors like OutOfMemoryException are not considered binding failures and are subject to retry.
• AppDomain.AppendPrivatePath and ClearPrivatePath are deprecated. These APIs allow the application to dynamically change where the .NET Framework looks for assemblies to satisfy binding requests. These APIs are subject to race conditions and cannot be reliably used.

The v1.x binder was arbitrarily assigning binding preferences between two methods. For example, in v1.x if you swap the declared order of M0 (or M2) the binder will bind to the other method.

using System;
using System.Reflection;

class C
{
public void M0(object arg) { Console.WriteLine(2); }
public void M0(ref int arg) { Console.WriteLine(1); }

public void M1(ref object arg) { Console.WriteLine(2); }
public void M1(ref int arg) { Console.WriteLine(1); }
}
class Repro
{
static void Main()
{
// Result depends on the order the M0 overloads were declared
typeof(C).InvokeMember("M0",
BindingFlags.Public | BindingFlags.Instance | BindingFlags.InvokeMethod,
null, new C(), new object[] { 100 }, null);

// Result depends on the order the M1 overloads were declared
typeof(C).InvokeMember("M1", BindingFlags.Public | BindingFlags.Instance |
BindingFlags.InvokeMethod,
null, new C(), new object[] { 100 }, null);
}
}

Now, in v2.0, we throw an AmbiguousMatchException and inform the developer about the ambiguous match rather than leave in the arbitrary, and prone-to-change, behavior of previous releases.

If you pass in an array with one or more null entries into the API WaitHandle.WaitAll, in V1.1 this would throw an ArgumentNullException. In V2.0 this will throw a FatalExcecutionException, which cannot be caught and will tear down the process.

This happens because of a coding error. There is code to detect this situation and throw an exception, but some code earlier in the routine puts the array in a special wrapper that dereferences all the entries in it, which causes an AV in the VM, which tears down the process because that is the policy for failures of this type.

Applications dependent on the implementation of private fields could be broken when the private field's type changes.

For instance, an app could use reflection to obtain a private field and try casting that field to an illegal type.

This has broken applications such as the Enterprise Library configuration tool, where the field type used to be safely cast to an IntPtr value, but since it has changed to a SafeHandle in v2.0 can no longer be cast this way.

The StackTrace dump in Environment and Exception had an error in V1.1 where it did not report nested type names correctly:

namespace A {
public class B {
public class C {
public string f() {
return new StackTrace().ToString
();
}
}
}
}

In V1.1 this reported "A.C.f()". In V2.0 the error was corrected to report "A.B.C.f()". Test code that baselines stack traces or code that cracked the text output of stack traces could have taken a dependency on the original bug.

Put a catch block at the top of your non-main thread, threadpool workitem, or finalizer. (Or else fix the bug that led to the exception.)

Alternatively, in the section of the application's config file, add the following:<legacyUnhandledExceptionPolicy enabled="1"/>

Class .ctor ordering for beforefieldinit types is not specified by ECMA. ECMA only specifies that runtimes will execute clas .ctors on beforefieldinit types some time prior to any access to a type's static field. It does not offer any guarantees regarding when exactly this might happen and in which order exactly. This of course never stopped anybody to (unknowingly) take a dependency on unspecified behavior if the behavior is consistent on the same runtime version.

Note: C# automatically marks types this way when static fields are initialized using "=" syntax like in "static int i = 5;". VB.NET programs are not affected since they use precise class .ctor semantics always.

An FxCop rule has been added to help detect the invalid pattern.

A thread can save the state of ReaderWriterLock into a LockCookie, and restore the state in the future using the saved LockCookie. A LockCookie can be serialized to a buffer. The buffer can be modified and then deserialized to a LockCookie. Now a thread can use the new LockCookie to restore the state of ReaderWriterLock. A LockCookie for one ReaderWriterLock can be used to restore a different ReaderWriterLock.

In addition, a LockCookie contains thread information about the thread to which a lock belongs. Serializing to a buffer, changing the thread id, and deserializing could open possible desynchronization attacks against arbitrarily chosen threads, provided that their identifiers were known or could be guessed

In 1.1 the three values in the enum System.Diagnostics.PerformanceCounterPermissionAccess were Browse:6, Instrument:2, Admin:14. These have been changed to browse:3, Instrument:1, Admin:7

Additionally, the value of System.AttributeTargets.All has changed from 16383, to 32767

IntPtr oldHandle = waitHandle.Handle; Win32.CloseHandle( oldHandle ); waitHandle.Handle = Win32.CreateEvent( );

From V1.1 to V2.0 the WaitHandle class has changed. It now points to a SafeWaitHandle, not to the raw handle.

Getting the handle and closing it is bad. The SafeWaitHandle still tracks the old handle and it will try to close it at finalize time. This means that the handle can be attempted to be closed twice.

The .NET Framework 1.1 C# compiler in certain cases generates code that calls virtual methods non-virtually. One of these cases is where there is a switch statement based on strings where the number of cases is greater than 8. In .NET Framework 2.0 a change was made to the verification rules for security purposes to disallow calling virtual methods non-virtually (in most cases). Therefore when code that was compiled using the .NET Framework 1.1 compiler is executed against .NET Framework 2.0, and the compiled MSIL binary contains a normal CALL (instead of a CALLVIRT) to a virtual method, and the assembly is running in partial trust, the MSIL needs to be verified, and it fails verification because of the new rule. The failure shows up as a System.Security.VerificationException.

The next broad release of the .NET Framework (2.0 SP1 or Orcas) has a workaround for this issue. The workaround is to modify the machine.config file for the machine where the application is running by replacing the default "<runtime />" with:

<runtime>
    <legacyVirtualMethodCallVerification enabled="1" />
</runtime>