.NET Matters
ThreadPoolWait and HandleLeakTracker
Stephen Toub
Code download available at:
NETMatters0410.exe
(172 KB)
Browse the Code Online
Q I have a few tasks that I want to run asynchronously and simultaneously, and I want to wait for them all to finish before continuing. I've seen samples showing how to do this using Thread.Join, but those seem to conflict with the general recommendation to use the ThreadPool whenever possible rather than spinning up new threads. Is there a way to wait for operations queued to the ThreadPool to finish execution?
Q I have a few tasks that I want to run asynchronously and simultaneously, and I want to wait for them all to finish before continuing. I've seen samples showing how to do this using Thread.Join, but those seem to conflict with the general recommendation to use the ThreadPool whenever possible rather than spinning up new threads. Is there a way to wait for operations queued to the ThreadPool to finish execution?
A As you point out, the classic solution to this problem is to spawn a new thread for each operation, start each thread executing, and then join the main thread to each of the child threads. Such a solution might look something like the following:
ArrayList threads = new ArrayList();
for(int i=0; i<numOperationsToPerform; i++)
{
Thread t = new Thread(new ThreadStart(DoSomething));
t.Start();
threads.Add(t);
}
foreach(Thread t in threads) t.Join();
A As you point out, the classic solution to this problem is to spawn a new thread for each operation, start each thread executing, and then join the main thread to each of the child threads. Such a solution might look something like the following:
ArrayList threads = new ArrayList();
for(int i=0; i<numOperationsToPerform; i++)
{
Thread t = new Thread(new ThreadStart(DoSomething));
t.Start();
threads.Add(t);
}
foreach(Thread t in threads) t.Join();
Join operations block the calling thread until the target thread terminates (or until the specified time elapses if one of the other overloads of Join is used that accepts timeout parameters). Thus, you need to start all of the new threads before joining to any of them; otherwise, if you were to instead execute code like the following, you'd inadvertently be serializing all of the tasks:
for(int i=0; i<numOperationsToPerform; i++)
{
Thread t = new Thread(new ThreadStart(DoSomething));
t.Start();
t.Join();
}
Spinning up new threads for every task may actually be the correct thing to do in your situation. But suppose you code this up and measure it, only to find that the use of new threads is the cause of performance problems (which could be the case if the number of simultaneous tasks is quite large). The .NET ThreadPool was created for situations like this.
A thread pool enables you to use threads more efficiently by providing your application with a set of worker threads that are managed by the system. All you have to do is use the ThreadPool.QueueUserWorkItem method to queue a WaitCallback delegate for execution, and the method referenced by the delegate will execute when a thread from the pool becomes available (or immediately if the thread pool has free threads available). Of course, since QueueUserWorkItem is asynchronous to your method (meaning that it returns immediately and most likely before your method has finished executing), as a user of the ThreadPool you don't know on which thread your method will execute, making it impossible to correctly join to it. Another solution is needed.
What you really need is a way for the main thread to keep track of all of the work items it's spawned and then for each work item to notify the main thread of its completion. A straightforward way to do this is to use an event object, such as System.Threading.ManualResetEvent. ManualResetEvent allows one thread to wait until another thread puts the event into a signaled state. Thus, the main thread can create an event, pass it to the worker thread, and wait for that event to be signaled. The worker thread can then signal the event when it has finished executing, as shown in the first half of Figure 1.
Figure 1 Waiting on ThreadPool Work Items
Waiting on One Work Item
static void DoSomething(object state)
{
ManualResetEvent mre = (ManualResetEvent)state;
try
{
// ... do work ...
}
finally
{
mre.Set();
}
}
static void Main()
{
ManualResetEvent mre = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(new WaitCallback(DoSomething), mre);
mre.WaitOne();
•••
}
static void DoSomething(object state)
{
ManualResetEvent mre = (ManualResetEvent)state;
try
{
// ... do work ...
}
finally
{
mre.Set();
}
}
static void Main()
{
ManualResetEvent mre1 = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(new WaitCallback(DoSomething), mre1);
ManualResetEvent mre2 = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(new WaitCallback(DoSomething), mre2);
ManualResetEvent mre3 = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(new WaitCallback(DoSomething), mre3);
WaitHandle.WaitAll(new WaitHandle[]{mre1, mre2, mre3});
•••
}
The second half of Figure 1 shows this same approach for supporting waiting on multiple work items. Each work item is given its own ManualResetEvent to signal on completion, and the main thread uses WaitHandle.WaitAll to block until all of the work item events have been set.
I've formalized this approach by writing the class shown in
Figure 2. Using this class, you can execute code like the following:
ThreadPoolWait tpw = new ThreadPoolWait();
tpw.QueueUserWorkItem(new WaitCallback(DoSomething1));
tpw.QueueUserWorkItem(new WaitCallback(DoSomething2));
tpw.QueueUserWorkItem(new WaitCallback(DoSomething3));
tpw.WaitOne();
Figure 2 First Attempt at ThreadPoolWait
public class ThreadPoolWait : IDisposable
{
private ArrayList _list = new ArrayList();
public void QueueUserWorkItem(WaitCallback callback)
{
QueueUserWorkItem(callback, null);
}
public void QueueUserWorkItem(WaitCallback callback, object state)
{
ThrowIfDisposed();
ManualResetEvent waitHandle = new ManualResetEvent(false);
_list.Add(waitHandle);
QueuedCallback qc =
new QueuedCallback(callback, state, waitHandle);
ThreadPool.QueueUserWorkItem(new
WaitCallback(qc.HandleCallback));
}
private class QueuedCallback
{
private WaitCallback _callback;
private object _state;
private ManualResetEvent _waitHandle;
public QueuedCallback(WaitCallback callback,
object state, ManualResetEvent waitHandle)
{
_callback = callback;
_state = state;
_waitHandle = waitHandle;
}
public void HandleCallback(object state)
{
try { _callback(_state); }
finally { _waitHandle.Set(); }
}
}
public bool WaitOne() { return WaitOne(-1, false); }
public bool WaitOne(TimeSpan timeout, bool exitContext)
{
return WaitOne(timeout.TotalMilliseconds, exitContext);
}
public bool WaitOne(int millisecondsTimeout, bool exitContext)
{
ThrowIfDisposed();
WaitHandle [] handles =
(WaitHandle[])_list.ToArray(typeof(WaitHandle));
return WaitHandle.WaitAll(
handles, millisecondsTimeout, exitContext);
}
private void ThrowIfDisposed()
{
if (_list == null)
throw new ObjectDisposedException(GetType().Name);
}
public void Dispose()
{
if (_list != null)
{
foreach(ManualResetEvent handle in _list)
{
((IDisposable)handle).Dispose();
}
_list.Clear();
}
}
}
Under the covers, ThreadPoolWait handles all of the eventing logic necessary for the main thread to block until all of the worker threads have completed execution. When the ThreadPoolWait.QueueUserWorkItem method is called, it creates a new ManualResetEvent and stores it into an internal list of events. It then wraps this event and all of the parameters to the method into a simple state class. This state will be passed to the method invoked on a ThreadPool thread (methods to be executed on a ThreadPool thread must be compatible with the WaitCallback delegate, and thus must take a single object parameter).
ThreadPoolWait.QueueUserWorkItem finishes by delegating to ThreadPool.QueueUserWorkItem, but rather than passing to this method the caller-supplied WaitCallback, instead a new WaitCallback is created that wraps the private QueuedCallback.HandleCallback method (actually, HandleCallback is marked as public, but since its containing class, QueuedCallback, is private to ThreadPoolWait, HandleCallback is limited by the accessibility of its containing class). When executed by the ThreadPool, this method will raise the user-supplied callback with the user-supplied state, and upon completion will raise the ManualResetEvent created in QueueUserWorkItem.
This class works well in some situations, but unfortunately it has a few implementation drawbacks. First, each queued user work item is assigned its own ManualResetEvent. As a result, queueing hundreds of work items means hundreds of ManualResetEvents will be created. Second, and more importantly, WaitHandle.WaitAll has an upper bound to the number of handles on which it can wait; on my machine running Windows® XP, attempting to wait on more than 64 events results in a NotSupportedException being thrown. This means that with the implementation shown in Figure 1, either a maximum of 64 work items can be queued, or a different mechanism than WaitHandle.WaitAll must be used in order to monitor all of the events.
Another viable solution (but one that has similar problems to the approach just described) is to use delegates and a delegate's BeginInvoke method. As opposed to a delegate's Invoke method, which runs the referenced method on the current thread, BeginInvoke queues the method to be run on a ThreadPool thread. The IAsyncResult returned from BeginInvoke exposes a WaitHandle from its AsyncWaitHandle property (most likely a ManualResetEvent, although that is an internal implementation detail on which you shouldn't rely). Just as with the previously discussed approach, WaitHandle.WaitAll can be used to wait on this WaitHandle, which won't be signaled until the queued method has terminated, either successfully or by exception. This still suffers from the WaitHandle.WaitAll limit mentioned earlier.
A better solution that solves both of these problems is shown in Figure 3. Rather than creating one ManualResetEvent per work item, a single ManualResetEvent is used for the entire system. Whether or not the completion of a work item requires that this event be signaled is based on a counter that keeps track of how many work items that have been scheduled still have not finished executing. When a work item is scheduled, the count is incremented using System.Threading.Interlocked.Increment. And when a work item finishes, it decrements the count using Interlocked.Decrement; if the new value of the counter is zero, then the event is signaled.
Figure 3 Better Implementation of ThreadPoolWait
public class ThreadPoolWait : IDisposable
{
private int _remainingWorkItems = 1;
private ManualResetEvent _done = new ManualResetEvent(false);
public void QueueUserWorkItem(WaitCallback callback)
{
QueueUserWorkItem(callback, null);
}
public void QueueUserWorkItem(WaitCallback callback, object state)
{
ThrowIfDisposed();
QueuedCallback qc = new QueuedCallback();
qc.Callback = callback;
qc.State = state;
Interlocked.Increment(ref _remainingWorkItems);
ThreadPool.QueueUserWorkItem(
new WaitCallback(HandleWorkItem), qc);
}
public bool WaitOne() { return WaitOne(-1, false); }
public bool WaitOne(TimeSpan timeout, bool exitContext)
{
return WaitOne((int)timeout.TotalMilliseconds, exitContext);
}
public bool WaitOne(int millisecondsTimeout, bool exitContext)
{
ThrowIfDisposed();
DoneWorkItem(); // to offset initial value of _remainingWorkItems
bool rv = _done.WaitOne(millisecondsTimeout, exitContext);
if (rv)
{
_remainingWorkItems = 1;
_done.Reset();
}
else Interlocked.Increment(ref _remainingWorkItems);
return rv;
}
private void HandleWorkItem(object state)
{
QueuedCallback qc = (QueuedCallback)state;
try { qc.Callback(qc.State); }
finally { DoneWorkItem(); }
}
private void DoneWorkItem()
{
if (Interlocked.Decrement(ref _remainingWorkItems) == 0)
_done.Set();
}
private class QueuedCallback
{
public WaitCallback Callback;
public object State;
}
private void ThrowIfDisposed()
{
if (_done == null)
throw new ObjectDisposedException(GetType().Name);
}
public void Dispose()
{
if (_done != null)
{
((IDisposable)_done).Dispose();
_done = null;
}
}
}
Notice that the counter's value is initially set to one rather than to zero, which may seem odd. This is done to work around the problem of a work item finishing before all of the work items have been queued; in such a scenario, the event could be signaled before all work items have completed. As a workaround, the counter is initially set to one to ensure that the counter remains above zero until WaitOne is called. WaitOne begins by removing this extra value so that the count accurately represents the remaining work items to be completed. Note, too, that by the time WaitOne is called, all of the work items may have completed and thus the call WaitOne makes to decrement the counter may, in fact, bring the counter to zero. Thus, WaitOne has to perform the same check as the work item, signaling the ManualResetEvent if there is no outstanding work to be done.
Q I have some code that works with IDisposable objects, many of which hold onto unmanaged handles of one sort or another. In my unit tests I want to make sure that I've disposed of any object that requires disposal as soon as possible rather than forcing the finalizer to clean it up. What are some options available to me?
Q I have some code that works with IDisposable objects, many of which hold onto unmanaged handles of one sort or another. In my unit tests I want to make sure that I've disposed of any object that requires disposal as soon as possible rather than forcing the finalizer to clean it up. What are some options available to me?
A One common practice used both outside and inside of Microsoft is to take advantage of finalization for notification that an object wasn't cleaned up in a timely manner. As discussed in the
.NET Matters column in the May 2004 edition of
MSDN®Magazine finalizers should only be implemented in a release version when they're absolutely necessary to ensure that all unmanaged resources are released properly.
A One common practice used both outside and inside of Microsoft is to take advantage of finalization for notification that an object wasn't cleaned up in a timely manner. As discussed in the
.NET Matters column in the May 2004 edition of
MSDN®Magazine finalizers should only be implemented in a release version when they're absolutely necessary to ensure that all unmanaged resources are released properly.
However, code can be added to a finalizer that alerts the developer as to when it's run (this should only be done in a debug version unless you have very special circumstances that warrant otherwise). In addition, for classes that implement IDisposable but that don't necessitate a finalizer (for example, a managed class that only holds references to other managed classes that implement IDisposable), a notification finalizer could be conditionally compiled into the class in debug builds.
As an example of classes that were implemented to follow this pattern, take a look at the finalizer for FileStream as coded in the Shared Source Common Language Infrastructure (SSCLI), which is available for download at
http://msdn.microsoft.com/net/sscli:
~FileStream()
{
if (_handleProtector != null) {
BCLDebug.Correctness(_handleProtector.IsClosed,
"You didn't close a FileStream & it got finalized. Name:\""
+ _fileName + "\"");
Dispose(false);
}
}
If _handleProtector is not null, then the FileStream was not Disposed, and a debug assert is raised. Of course, calls to BCLDebug.Correctness are only compiled into _DEBUG builds, as dictated by the System.Diagnostics.ConditionalAttribute applied to the Correctness method.
This solution works very well when you're writing the class in question. But what if someone else has written the class and compiled it, and now you're using the assembly they've provided to you? In such a situation, certain types of leaks are easier to discover than others. For example, the base class shown in Figure 4 can be used to help track down native handle leaks using a very simple process: take a snapshot of the handle count before you run the leaky code in question, run the code, take a snapshot afterwards, and then compare the before and after numbers. This approach isn't foolproof and in some situations could result in both false positives and negatives, but in the general case it proves to be a very useful tactic while debugging.
Figure 4 Tracking Handle Leaks
public abstract class HandleLeakTracker : IDisposable
{
private bool _runGCAtDispose;
private int _count;
protected HandleLeakTracker() : this(true, false) {}
protected HandleLeakTracker(bool runGCNow, bool runGCAtDispose)
{
if (runGCNow) RunGC();
_runGCAtDispose = runGCAtDispose;
_count = HandleCount;
}
private void RunGC()
{
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
}
void IDisposable.Dispose()
{
if (_runGCAtDispose) RunGC();
int newCount = HandleCount;
if (_count != newCount) Assert(_count, newCount);
}
protected abstract int HandleCount { get; }
protected virtual void Assert(int oldCount, int newCount)
{
Debug.Fail("Possible Handle Leak", "Old Count: " + oldCount +
Environment.NewLine + "New Count: " + newCount);
}
}
How do you use the code in
Figure 4? The HandleLeakTracker class shown is an abstract base class, so it is assumed you will derive your own specialized class from this in order to track the specific type of resource in which you're interested. For example, if I want to ensure that a specific section of code does not leak a handle opened by the process, I can derive a new class from HandleLeakTracker. This class overrides the HandleCount abstract method and returns the result of calling Process.HandleCount for the current Process:
public class ProcessHandleLeakTracker : HandleLeakTracker {
protected override int HandleCount {
get {
using (Process p = Process.GetCurrentProcess()) {
return p.HandleCount;
}
}
}
}
Since HandleLeakTracker implements IDisposable, I can surround the leaky code in question with a using block as follows:
using(new ProcessHandleLeakTracker())
{
StreamWriter w = new StreamWriter(@"C:\test.txt");
w.WriteLine("Hello, World!");
}
The constructor of HandleLeakTracker stores the current HandleCount. The code in the using block runs, and then the Dispose method (executed from the finally block generated by the C# compiler as part of the using semantics) retrieves the new handle count and performs the comparison. The virtual Assert method uses Debug.Fail to signal any discrepancies.
Note, too, that both the constructor and the Dispose method can call HandleLeakTracker.RunGC depending on the Boolean parameters specified in the constructor. By default, a garbage collection is run before the handle count is initially taken so that a garbage collection that takes place while the code in question is running won't throw off the numbers. Of course, even with this precaution, situations arise which can result in a skewed handle count, such as a multithreaded application in which another thread is creating and destroying handles while the leak detection code is executing, or the garbage collector running arbitrarily during the using block and leading you into a false sense of security. The beauty of using this in your unit tests, though, is that while it won't always point out problems that exist, there's a good chance it'll help you find more than you otherwise would have discovered.
Other variants of this class are possible, such as a HandleLeakTracker-derived class that uses the GetGuiResources function exposed from User32.dll to track Graphical Device Interface (GDI) and User object leaks, shown here:
public class GuiHandleLeakTracker : HandleLeakTracker {
protected override int HandleCount
{
get {
using (Process p = Process.GetCurrentProcess())
return GetGuiResources(p.Handle, 0) +
GetGuiResources(p.Handle, 1);
}
}
[DllImport("User32")]
private static extern int GetGuiResources(
IntPtr hProcess, int uiFlags);
}
The base class could even be modified to support snapshot-capable resources besides handles, or could be integrated with a kernel-mode driver capable of reporting exactly which handles are open, thus allowing the class to determine not only how many handles are erroneously open but also to pinpoint exactly which ones were leaked. As usual, the possibilities are practically endless, and I'd be interested to hear what other solutions you find.
Send your questions and comments to netqa@microsoft.com.
Stephen Toub is the Technical Editor for MSDN Magazine.