.NET Matters
Asynchronous HttpWebRequests, Interface Implementation, and More
Stephen Toub
Code download available at:
NETMatters0412.exe
(163 KB)
Browse the Code Online
Q My client and I are working on a client-side application that entails making HttpWebRequests to a server application in order to submit data. We're trying to limit the number of concurrent connections made by the client so we can limit the load placed on the server. Initially we tried making these requests from ThreadPool threads, but we continually received exceptions specifying that "There were not enough free threads in the ThreadPool object to complete the operation." So I have two questions. First, why doesn't the ThreadPool have enough threads; isn't the whole point of the ThreadPool that it should cause queued work items to block until there are threads available? And second, if we can't use the ThreadPool to do this, how can we throttle the number of concurrent connections requested?
Q My client and I are working on a client-side application that entails making HttpWebRequests to a server application in order to submit data. We're trying to limit the number of concurrent connections made by the client so we can limit the load placed on the server. Initially we tried making these requests from ThreadPool threads, but we continually received exceptions specifying that "There were not enough free threads in the ThreadPool object to complete the operation." So I have two questions. First, why doesn't the ThreadPool have enough threads; isn't the whole point of the ThreadPool that it should cause queued work items to block until there are threads available? And second, if we can't use the ThreadPool to do this, how can we throttle the number of concurrent connections requested?
A Excellent set of questions, and based on a quick Web search one that has stymied many developers. The first thing to be aware of is that in version 1.
x of the Microsoft
®.NET Framework, HttpWebRequest never makes synchronous requests. What do I mean by that? Take a look at the code for HttpWebRequest.GetResponse as coded in the
Shared Source CLI (SSCLI), shown here omitting the code that checks to see if the response was previously retrieved and that accounts for timeouts:
public override WebResponse GetResponse() {
•••
IAsyncResult asyncResult = BeginGetResponse(null, null);
•••
return EndGetResponse(asyncResult);
}
A Excellent set of questions, and based on a quick Web search one that has stymied many developers. The first thing to be aware of is that in version 1.
x of the Microsoft
®.NET Framework, HttpWebRequest never makes synchronous requests. What do I mean by that? Take a look at the code for HttpWebRequest.GetResponse as coded in the
Shared Source CLI (SSCLI), shown here omitting the code that checks to see if the response was previously retrieved and that accounts for timeouts:
public override WebResponse GetResponse() {
•••
IAsyncResult asyncResult = BeginGetResponse(null, null);
•••
return EndGetResponse(asyncResult);
}
As you can see, HttpWebRequest.GetResponse is simply a wrapper around the pairing of BeginGetResponse and EndGetResponse. These operate asynchronously, meaning that BeginGetResponse makes the actual HTTP request from a different thread than the one from which it was called, and EndGetResponse blocks until the request has completed. The net result of this is that HttpWebRequest queues a work item to the ThreadPool for every outbound request. So, we know that HttpWebRequest uses threads from the ThreadPool, but why does that cause a problem if the call to GetResponse is made from a ThreadPool thread? Deadlocks.
As you pointed out in your question, the ThreadPool will queue work items which will wait until threads are available to process them. For discussion's sake, let's say that the ThreadPool is a pool of only one thread (it's of course much larger by default). You queue a method that calls HttpWebRequest.GetResponse, and this method starts executing on the sole thread in the ThreadPool. The GetResponse method calls BeginGetResponse, which in turn queues a work item to the ThreadPool, and then calls EndGetResponse to wait for that work item to be completed. Unfortunately, that'll never happen. The work item BeginGetResponse queued won't execute until a ThreadPool thread becomes available, but the only thread in the ThreadPool is currently executing the call to EndGetResponse waiting for the HTTP request to complete. Deadlock. "But," you say, "this is a contrived example with only one thread in the pool, and as you said there are many more threads in the pool than that." True, but let's expand the example. What happens if there are two threads in the pool, and you quickly queue two methods that call GetResponse before either has a chance to queue their work items? Same problem, deadlock. What happens if there are 25 threads in the pool and you quickly queue 25 of these methods? Again, deadlock. See the problem?
As a workaround for this, the Framework team implemented the exception you're seeing. At the end of BeginGetResponse, just before the work item is queued, System.Net.Connection.IsThreadPoolLow is used to query how many worker threads are available in the pool. If the number is too low (less than two), an InvalidOperationException is thrown.
The good news is that in the .NET Framework 2.0, synchronous requests made with HttpWebRequest.GetResponse are truly synchronous, and this problem goes away, allowing you to queue methods that call GetResponse to your heart's content. The less-than-good news is that you're stuck with this problem in version 1.x. One solution, as you point out, is to explicitly throttle the number of work items that you have queued to or are executing in the ThreadPool at any one time. To implement this approach, you need a way to keep track of how many work items are currently outstanding and block new requests when the predetermined limit has been reached.
Semaphores are synchronization primitives that maintain a count between zero and some maximum value. Operations are provided that raise and lower the count, the trick being that attempting to lower the count when the count is already zero causes the calling thread to block until another thread comes along and increases the count. This makes semaphores great for a few different purposes. The first, and probably one of the most common scenarios discussed in a college operating systems class, is a producer/consumer model where producer threads generate products later used by consumer threads. The second, and arguably a superset of the first, is the controlling of access to shared resources that can support only a limited number of users. In such a scenario, the semaphore acts as a guard, blocking users' attempts to access a resource while the maximum allowed number of users are currently accessing it. Sounds perfect for this scenario, right?
In fact, it is. Unfortunately, version 1.x of the .NET Framework does not include a semaphore implementation. However, putting together a simple wrapper for the Win32® semaphore object requires only a handful of code, as shown in Figure 1 (note that the .NET Framework version 2.0 does include a Semaphore implementation, and a much more complete one than the class shown here). The DllImport attribute is used with P/Invoke to wrap CreateSemaphore (the Win32 function exposed from kernel32.dll and used to instantiate a semaphore object) and ReleaseSemaphore (also exposed from kernel32.dll and used to increase the count of the semaphore). A semaphore object is referred to using the handle returned by CreateSemaphore, which makes this a perfect opportunity to derive my Semaphore class from WaitHandle and to use the functionality it provides (WaitOne, for example) for waiting on synchronization objects. Using the base class's WaitOne will cause the semaphore count to be decremented, blocking if the count is already at zero and until the count is increased or until a specified timeout elapses.
Figure 1 .NET Semaphore That Wraps Win32 Semaphore
public sealed class Semaphore : WaitHandle
{
public Semaphore() : this(1, 1) {}
public Semaphore(int initialCount, int maximumCount)
{
if (initialCount < 0 || initialCount > maximumCount)
throw new ArgumentOutOfRangeException("initialCount");
if (maximumCount < 1)
throw new ArgumentOutOfRangeException("maximumCount");
IntPtr h = CreateSemaphore(
IntPtr.Zero, initialCount, maximumCount, null);
if (h == WaitHandle.InvalidHandle || h == IntPtr.Zero)
throw new Win32Exception();
Handle = h;
}
public void ReleaseOne()
{
int previousCount;
if (!ReleaseSemaphore(Handle, 1, out previousCount))
throw new Win32Exception();
}
[DllImport("kernel32.dll", SetLastError=true)]
private static extern IntPtr CreateSemaphore(
IntPtr lpSemaphoreAttributes, int lInitialCount,
int lMaximumCount, string lpName);
[DllImport("kernel32.dll", SetLastError=true)]
private static extern bool ReleaseSemaphore(
IntPtr hSemaphore, int lReleaseCount, out int lpPreviousCount);
}
With this semaphore, it's now fairly straightforward to create a class that throttles items queued to the ThreadPool. Such an implementation is shown in
Figure 2. The semaphore is initially configured to have an initial and maximum count equal to the maximum number of requests you want to allow to execute simultaneously. When ThreadPoolThrottle.QueueUserWorkItem is invoked, the semaphore's WaitOne method is called to decrement the count and block if the maximum number of requests are currently executing. Just as with the ThreadPoolWait class from the
October 2004 installment of .NET Matters, the user-specified WaitCallback and its associated state are packed into a new state object which is then queued to the ThreadPool as the state for a private method, HandleWorkItem. When invoked by the ThreadPool, HandleWorkItem invokes the user-specified delegate with the user-specified state. It then increments the semaphore to signal that this unit of work has completed execution, allowing a blocked thread to wake up and continue with its request. Assuming nothing else is using the pool at the same time (thus decreasing the number of available threads), a class like ThreadPoolThrottle can be used to successfully throttle the number of work items queued.
Figure 2 Throttling Requests to the ThreadPool
public class ThreadPoolThrottle : IDisposable
{
private Semaphore _throttle;
public ThreadPoolThrottle(int maximumAllowed)
{
if (maximumAllowed < 1)
throw new ArgumentOutOfRangeException("maximumAllowed");
_throttle = new Semaphore(maximumAllowed,maximumAllowed);
}
public void QueueUserWorkItem(WaitCallback callback)
{
QueueUserWorkItem(callback, null);
}
public void QueueUserWorkItem(WaitCallback callback, object state)
{
if (_throttle == null)
throw new ObjectDisposedException(this.GetType().FullName);
if (callback == null)
throw new ArgumentNullException("callback");
_throttle.WaitOne();
try
{
QueuedCallback qc = new QueuedCallback();
qc.Callback = callback;
qc.State = state;
ThreadPool.QueueUserWorkItem(
new WaitCallback(HandleWorkItem), qc);
}
catch
{
_throttle.ReleaseOne();
throw;
}
}
private void HandleWorkItem(object state)
{
QueuedCallback qc = (QueuedCallback)state;
try { qc.Callback(qc.State); }
finally { _throttle.ReleaseOne(); }
}
private class QueuedCallback
{
public WaitCallback Callback;
public object State;
}
public void Dispose()
{
if (_throttle != null)
{
((IDisposable)_throttle).Dispose();
_throttle = null;
}
}
}
Q I'm writing a tool that uses reflection. Given a method on a class, I need to figure out which interface method that class method implements. If it doesn't implement an interface, I need to know that, too. Is this possible?
Q I'm writing a tool that uses reflection. Given a method on a class, I need to figure out which interface method that class method implements. If it doesn't implement an interface, I need to know that, too. Is this possible?
A First, the question is a bit misleading. You seem to be implying that a method on a class can only be used to implement one method on one interface, but that in fact is incorrect. A class method can be used to implement methods on multiple interfaces, or even multiple methods on the same interface, though not all languages support this functionality. Take the following C# sample as an example:
interface SomeInterface1 {
void Method1();
}
interface SomeInterface2 {
void Method1();
}
class SomeClass : SomeInterface1, SomeInterface2 {
public void Method1(){}
}
A First, the question is a bit misleading. You seem to be implying that a method on a class can only be used to implement one method on one interface, but that in fact is incorrect. A class method can be used to implement methods on multiple interfaces, or even multiple methods on the same interface, though not all languages support this functionality. Take the following C# sample as an example:
interface SomeInterface1 {
void Method1();
}
interface SomeInterface2 {
void Method1();
}
class SomeClass : SomeInterface1, SomeInterface2 {
public void Method1(){}
}
Here, the method SomeClass.Method1 implements two interface methods, SomeInterface1.Method1 and SomeInterface2.Method1, which both use implicit interface implementations. The C# language provides two ways in which a class method can be mapped to an interface method. One way is through explicit interface member implementation, indicated by naming the class member with both the name of the interface and the name of the interface method, as such:
class SomeClass : SomeInterface1 {
void SomeInterface1.Method1() {}
}
This causes the implementation to be exposed publicly but only through the interface. The second way, implicit implementation, is done by matching nonstatic public methods to interface methods by name, type, and formal parameter lists. As such, whereas in C# it's possible (as shown previously) to have one class method implement a method on two different interfaces, it isn't possible in C# to have one class method implement two different interface methods from the same interface. This is because it's invalid to have two methods on the same interface that have the same name, type, and formal parameters, a situation which would be required for a single class method to map to both. Visual Basic
® .NET solves this problem by forcing the developer to explicitly state what interface methods a class method is implementing. Thus, in Visual Basic .NET it is possible to have one class method implement two different interface methods, both from the same interface, as shown in the following code snippet:
Interface SomeInterface
Sub Method1()
Sub Method2()
End Interface
Class SomeClass
Implements SomeInterface
Public Sub SomeMethod() Implements _
SomeInterface.Method1, SomeInterface.Method2
Console.WriteLine("SomeMethod called.")
End Sub
End Class
Here, SomeClass.SomeMethod actually implements both SomeInterface.Method1 and SomeInterface.Method2.
With that said, back to the core of your question. Yes, it is possible using reflection to determine what interface methods a given class method implements, and one possible solution for doing so is shown in Figure 3.
Figure 3 Determining the Interfaces a Method Implements Implements
public static Type[] GetImplementedInterfaces(MethodInfo mi)
{
ArrayList list = new ArrayList();
Type reflectedType = mi.ReflectedType;
foreach(Type ift in reflectedType.GetInterfaces())
{
foreach(MethodInfo target in
reflectedType.GetInterfaceMap(ift).TargetMethods)
{
if (target.Equals(mi))
{
list.Add(ift);
break;
}
}
}
return (Type[])list.ToArray(typeof(Type));
}
The Type class exposes the GetInterfaceMap method which, when passed a Type representing an interface, is used to return a mapping that denotes how that interface is mapped into the actual methods on the class that implements that interface. InterfaceMap, the type returned by GetInterfaceMap, exposes two important fields: TargetMethods and InterfaceMethods. These fields store arrays, the former being an array of MethodInfo objects representing the methods on the type that implement interface methods, and the latter being the interface methods that map to the former array. So for example, the method represented by the MethodInfo at TargetMethods[2] implements the interface method represented by the MethodInfo stored at InterfaceMethods[2].
My GetImplementedInterfaces method shown in
Figure 3 takes a MethodInfo representing the method in question, and it returns an array of Type instances, each of which represents one interface that method is used to implement. First, the method's ReflectedType is retrieved, which is necessary in order to retrieve all of the interfaces that type implements as well as the interface mapping for that type's methods. You might notice that the MemberInfo class (from which MethodInfo derives) exposes a DeclaringType property in addition to ReflectedType. It's important that I not use DeclaringType for this purpose. The difference is that DeclaringType returns the type that declares this member, whereas ReflectedType returns the type that was used to retrieve the given MethodInfo. When are they different, and why does the distinction matter? For every interface implemented by the method's container class, I need to loop through each of the target methods in the interface map and see if the specified method matches any of those. If it does match, then the specified method is in the interface map for the given interface, which means that the method is used to implement that interface. However, there are certain situations in which you can obtain a MethodInfo for an interface-implementing method but where that MethodInfo won't match any MethodInfo in the TargetMethods array. Specifically, this can happen when the declaring type and the reflected type differ. Consider the following example:
interface SomeInterface {
void Method1();
}
class Base : SomeInterface {
public void Method1() {}
}
class Derived : Base {}
Derived.Method1 is actually implemented on the Base class. As a result, the MethodInfo retrieved using
typeof(Derived).GetMethod("Method1")
will be a different object than the MethodInfo retrieved using
typeof(Base).GetMethod("Method1")
If I were to use the GetInterfaceMap from the Base Type (which in this case is the type that declares Method1) to retrieve the interface mapping, but the MethodInfo had been retrieved using typeof(Derived), the MethodInfo won't match any of the objects in the TargetMethods array. Thus, it's important to call GetInterfaceMap on the same Type used to retrieve the specified MethodInfo, and that Type is the one returned from the MethodInfo's ReflectedType property.
Q I've been reading recently about COM threading models and how .NET fits into the picture. I know I can use the STAThreadAttribute and MTAThreadAttribute to initially set the threading model for the main thread of my application, and I know I can configure a thread using the Thread.ApartmentState property, but both of those techniques only seem to work before any calls to COM objects are made. After that, they appear to be immutable. Is that true? If so, what do I do if I need to use multiple COM objects, each of which uses a different threading model?
Q I've been reading recently about COM threading models and how .NET fits into the picture. I know I can use the STAThreadAttribute and MTAThreadAttribute to initially set the threading model for the main thread of my application, and I know I can configure a thread using the Thread.ApartmentState property, but both of those techniques only seem to work before any calls to COM objects are made. After that, they appear to be immutable. Is that true? If so, what do I do if I need to use multiple COM objects, each of which uses a different threading model?
A The Thread.ApartmentState property can be changed as often as you like until the first calls are made to a COM object. At that point, the threading model for the current thread becomes immutable and you won't be able to change it. If you need to use COM objects that require a different threading model than that of your current thread, the standard solution is to spin up a new thread, set its ApartmentState appropriately, and then execute the code that uses that COM object on that new thread. (For a refresher in COM threading models, I suggest reading Larry Osterman's blog post at
What are these "Threading Models" and why do I care?.)
A The Thread.ApartmentState property can be changed as often as you like until the first calls are made to a COM object. At that point, the threading model for the current thread becomes immutable and you won't be able to change it. If you need to use COM objects that require a different threading model than that of your current thread, the standard solution is to spin up a new thread, set its ApartmentState appropriately, and then execute the code that uses that COM object on that new thread. (For a refresher in COM threading models, I suggest reading Larry Osterman's blog post at
What are these "Threading Models" and why do I care?.)
To make this process a bit easier, I've implemented the class shown in Figure 4. ApartmentStateSwitcher exposes only one static method named Execute. This method takes a delegate to invoke, the parameters to pass to the delegate, and the ApartmentState required to execute this delegate. If the ApartmentState of the current thread matches the one requested by the user, the method simply invokes the delegate using the delegate's DynamicInvoke method, which provides for late-bound method invocation. However, if the apartment states differ, it invokes the delegate on a new thread with the ApartmentState set appropriately. To do this, it creates a small object that it can use to pass the state back and forth between the current and new thread. Into this object it stores the delegate and the parameters with which to invoke it.
Figure 4 ApartmentStateSwitcher
public sealed class ApartmentStateSwitcher
{
private Delegate _delegate;
private object[] _parameters;
private Exception _exc;
private object _rv;
private ApartmentStateSwitcher(){}
private void Run()
{
try { _rv = _delegate.DynamicInvoke(_parameters); }
catch(MemberAccessException exc) { _exc = exc; }
catch(TargetException exc) { _exc = exc; }
catch(TargetInvocationException exc) { _exc = exc; }
}
public static object Execute(
Delegate d, object[] parameters, ApartmentState state)
{
if (d == null) throw new ArgumentNullException("d");
if (state != ApartmentState.MTA && state != ApartmentState.STA)
throw new ArgumentOutOfRangeException("state");
if (Thread.CurrentThread.ApartmentState == state)
{
return d.DynamicInvoke(parameters);
}
else
{
ApartmentStateSwitcher switcher =
new ApartmentStateSwitcher();
switcher._delegate = d;
switcher._parameters = parameters;
Thread t = new Thread(new ThreadStart(switcher.Run));
t.ApartmentState = state;
t.IsBackground = Thread.CurrentThread.IsBackgound;
t.Start();
t.Join();
if (switcher._exc != null) throw switcher._exc;
return switcher._rv;
}
}
}
A new thread is then created with the appropriate ApartmentState, started, and immediately joined with the current thread, blocking the current thread until the new thread finishes execution. The main method for this new thread is Run, a private member of the class that executes the delegate with the supplied parameters. Any return value from the delegate's invocation is stored to the state class, as is any exception thrown during the process (in version 1.x of the .NET Framework, exceptions thrown on worker threads are simply eaten by the runtime, and in version 2.0, worker thread exceptions cause the AppDomain to shut down; in neither version are exceptions passed back to the main thread, so I have to do that manually). Back in the Execute method, when the worker thread finishes, the Thread.Join call completes and the state class is examined for an exception and a return value. If an exception exists, it is rethrown. If not, the return value from the delegate is returned.
Using this class is as easy as creating a delegate around whatever code you want executed (a process made much simpler with anonymous delegates in C# 2.0) and passing that delegate to ApartmentStateSwitcher.Execute, as shown here:
[MTAThread]
static void Main(string[] args) {
PrintCurrentState();
ApartmentStateSwitcher.Execute(
new ThreadStart(PrintCurrentState), null, ApartmentState.STA);
PrintCurrentState();
}
static void PrintCurrentState() {
Console.WriteLine("Thread apartment state: " +
Thread.CurrentThread.ApartmentState);
}
This code prints to the console:
Thread apartment state: MTA
Thread apartment state: STA
Thread apartment state: MTA
Send your questions and comments to netqa@microsoft.com.
Stephen Toub is the Technical Editor for MSDN Magazine.