Windows with C++ - Thread Pool Cancellation and Cleanup

By Kenny Kerr | October 2011

Kenny Kerr Cancellation and cleanup are notoriously difficult problems to solve when it comes to multithreaded applications. When is it safe to close a handle? Does it matter which thread cancels an operation? To make matters worse, some multithreaded APIs are not reentrant, potentially improving performance but also adding complexity for the developer.

I introduced the thread pool environment in last month’s column (msdn.microsoft.com/magazine/hh394144). One critical feature this environment enables is cleanup groups, and that’s what I’ll focus on here. Cleanup groups don’t attempt to solve all of the world’s cancellation and cleanup problems. What they do is make the thread pool’s objects and callbacks more manageable, and this can indirectly help to simplify the cancellation and cleanup of other APIs and resources as needed.

So far, I’ve only shown you how to use the unique_handle class template to automatically close work objects via the CloseThreadpoolWork function. (See the August 2011 column at msdn.microsoft.com/magazine/hh335066 for the details.) There are some limitations with this approach, however. If you want a say in whether pending callbacks are canceled or not, you need to call WaitForThreadpoolWorkCallbacks first. That makes two function calls multipliednumber of callback-generating objects in use by your application. If you opt to use TrySubmitThreadpoolCallback, you don’t even get the opportunity to do so and are left wondering how to cancel or wait for the resulting callback. Of course, a real-world application will likely have far more than just work objects. In next month’s column, I’ll start introducing the other thread pool objects that produce callbacks, from timers to I/O to waitable objects. Coordinating the cancellation and cleanup of all of these can quickly become a nightmare. Fortunately, cleanup groups solve these problems and more.

The CreateThreadpoolCleanupGroup function creates a cleanup group object. If the function succeeds, it returns an opaque pointer representing the cleanup group object. If it fails, it returns a null pointer value and provides more information via the GetLastError function. Given a cleanup group object, the CloseThreadpoolCleanupGroup function instructs the thread pool that the object may be released. I’ve mentioned this before in passing, but it bears repeating—the thread pool API does not tolerate invalid arguments. Calling CloseThreadpoolCleanupGroup or any of the other API functions with an invalid, previously closed or null pointer value will cause your application to crash. These are defects introduced by the programmer and should not require additional checks at run time. The unique_handle class template I introduced in my July 2011 column (msdn.microsoft.com/magazine/hh288076) takes care of these details with the help of a cleanup-group-specific traits class:

struct cleanup_group_traits
{
  static PTP_CLEANUP_GROUP invalid() throw()
  {
    return nullptr;
  }
 
  static void close(PTP_CLEANUP_GROUP value) throw()
  {
    CloseThreadpoolCleanupGroup(value);
  }
};
typedef unique_handle<PTP_CLEANUP_GROUP, cleanup_group_traits> cleanup_group;

I can now use the convenient typedef and create a cleanup group object as follows:

cleanup_group cg(CreateThreadpoolCleanupGroup());
check_bool(cg);

As with private pools and callback priorities, a cleanup group is associated with various callback-generating objects by means of an environment object. First, update the environment to indicate the cleanup group that will manage the lifetime of objects and their callbacks, like this:

environment e;
SetThreadpoolCallbackCleanupGroup(e.get(), cg.get(), nullptr);

At this point, you can add objects to the cleanup group that are then referred to as members of the cleanup group. These objects can also be individually removed from the cleanup group, but it’s more common to close all members in a single operation.

A work object can become a member of a cleanup group at creation time simply by providing the updated environment to the CreateThreadpoolWork function:

auto w = CreateThreadpoolWork(work_callback, nullptr, e.get());
check_bool(nullptr != w);

Notice that I didn’t use a unique_handle this time. The newly created work object is now a member of the environment’s cleanup group and its lifetime need not be tracked directly using RAII.

You can revoke the work object’s membership only by closing it, which can be done on an individual basis with the CloseThreadpoolWork function. The thread pool knows that the work object is a member of the cleanup group and revokes its membership before closing it. This ensures that the application doesn’t crash when the cleanup group later attempts to close all of its members. The inverse isn’t true: If you first instruct the cleanup group to close all of its members and then call CloseThreadpoolWork on the now invalid work object, your application will crash.

Of course, the whole point of a cleanup group is to free the application from having to individually close all of the various callback-generating objects it happens to be using. More importantly, it allows the application to wait for and optionally cancel any outstanding callbacks in a single wait operation rather than having to have an application thread wait and resume repeatedly. The CloseThreadpoolCleanupGroupMembers function provides all of these services and more:

bool cancel = ...
CloseThreadpoolCleanupGroupMembers(cg.get(), cancel, nullptr);

This function might appear simple, but in reality it performs a number of important duties as an aggregate on all of its members. First, depending on the value of the second parameter, it cancels any pending callbacks that haven’t yet begun to execute. Next, it waits for any callbacks that have already begun to execute and, optionally, any pending callbacks if you chose not to cancel them. Finally, it closes all of its member objects.

Some have likened cleanup groups to garbage collection, but I think this is a misleading metaphor. If anything, a cleanup group is more like a Standard Template Library (STL) container of callback-generating objects. Objects added to a group will not be automatically closed for any reason. If you fail to call CloseThreadpoolCleanupGroupMembers, your application will leak memory. Calling CloseThreadpoolCleanupGroup to close the group itself won’t help either. You should instead just think of a cleanup group as a way of managing the lifetime and concurrency of a group of objects. You can, of course, create multiple cleanup groups in your application to manage different groups of objects individually. It’s an incredibly useful abstraction—but it’s not magic, and care must be taken to use it correctly. Consider the following faulty pseudo-code:

environment e;
SetThreadpoolCallbackCleanupGroup(e.get(), cg.get(), nullptr);
 
while (app is running)
{
  SubmitThreadpoolWork(CreateThreadpoolWork(work_callback, nullptr, e.get()));
 
  // Rest of application.
}
 
CloseThreadpoolCleanupGroupMembers(cg.get(), true, nullptr);

Predictably, this code will use an unbounded amount of memory and will get slower and slower as system resources are exhausted.

In my August 2011 column, I demonstrated that the seemingly simple TrySubmitThreadpoolCallback function is rather problematic because there’s no simple way to wait for its callback to complete. This is because the work object is not actually exposed by the API. The thread pool itself, however, suffers no such restriction. Because TrySubmitThreadpoolCallback accepts a pointer to an environment, you can indirectly make the resulting work object a member of a cleanup group. In this way, you can use CloseThreadpoolCleanupGroupMembers to wait for or cancel the resulting callback. Consider the following improved pseudo-code:

environment e;
SetThreadpoolCallbackCleanupGroup(e.get(), cg.get(), nullptr);
 
while (app is running)
{
  TrySubmitThreadpoolCallback(simple_callback, nullptr, e.get());
 
  // Rest of application.
}
 
CloseThreadpoolCleanupGroupMembers(cg.get(), true, nullptr);

I could almost forgive a developer for thinking this is akin to garbage collection, because the thread pool automatically closes the work object created by TrySubmitThreadpoolCallback. Of course, this has nothing to do with cleanup groups. I described this behavior in my July 2011 column. The CloseThreadpoolCleanupGroupMembers function in this case is not initially responsible for closing the work object, but only for waiting for and possibly canceling callbacks. Unlike the previous example, this one will run indefinitely without using undue resources and still provide  predictable cancelation and cleanup. With the help of callback groups, TrySubmitThreadpoolCallback redeems itself, providing a safe and convenient alternative. In a highly structured application where the same callback is queued repeatedly, it would still be more efficient to reuse an explicit work object, but the convenience of this function can no longer be dismissed.

Cleanup groups provide one final feature to simplify your application’s cleanup requirements. Often, it’s not enough to simply wait for outstanding callbacks to complete. You might need to perform some cleanup task for each callback-generating object once you’re sure no further callbacks will execute. Having a cleanup group manage the lifetime of these objects also means that the thread pool is in the best position to know when such cleanup tasks should happen.

When you associate a cleanup group with an environment through the SetThreadpoolCallbackCleanupGroup function, you can also provide a callback to be executed for each member of the cleanup group as part of the CloseThreadpoolCleanupGroupMembers function’s process of closing these objects. Because this is an attribute of the environment, you can even apply different callbacks to different objects belonging to the same cleanup group. In the following example I create an environment for the cleanup group and cleanup callback:

void CALLBACK cleanup_callback(void * context, void * cleanup)
{
  printf("cleanup_callback: context=%s cleanup=%s\n", context, cleanup);
}
 
environment e;
SetThreadpoolCallbackCleanupGroup(e.get(), cg.get(), cleanup_callback);

The cleanup callback’s first parameter is the context value for the callback-generating object. This is the context value you specify when calling the CreateThreadpoolWork or TrySubmitThreadpoolCallback functions, for example, and it’s how you know which object the cleanup callback is being called for. The cleanup callback’s second parameter is the value provided as the last parameter when calling the CloseThreadpoolCleanupGroupMembers function.

Now consider the following work objects and callbacks:

void CALLBACK work_callback(PTP_CALLBACK_INSTANCE, void * context, PTP_WORK)
{
  printf("work_callback: context=%s\n", context);
}
 
void CALLBACK simple_callback(PTP_CALLBACK_INSTANCE, void * context)
{
  printf("simple_callback: context=%s\n", context);
}
 
SubmitThreadpoolWork(CreateThreadpoolWork(work_callback, "Cheetah", e.get()));
SubmitThreadpoolWork(CreateThreadpoolWork(work_callback, "Leopard", e.get()));
check_bool(TrySubmitThreadpoolCallback(simple_callback, "Meerkat", e.get()));

Which of these is not like the others? As much as the cute little Meerkat wants to be just like his neighboring big cats in southern Africa, he will simply never be one of them. What happens when the cleanup group members are closed as follows?

CloseThreadpoolCleanupGroupMembers(cg.get(), true, "Cleanup");

Very little is certain in multithreaded code. If the callbacks manage to execute before they’re canceled and closed, the following might be printed out:

work_callback: context=Cheetah
work_callback: context=Leopard
simple_callback: context=Meerkat
cleanup_callback: context=Cheetah cleanup=Cleanup
cleanup_callback: context=Leopard cleanup=Cleanup

A common mistake is to assume that the cleanup callback is called only for objects whose callbacks did not get an opportunity to execute. The Windows API is a bit misleading because it sometimes refers to the cleanup callback as a cancel callback, but this is not the case. The cleanup callback is simply called for every current member of the cleanup group. You could think of it as a destructor for cleanup group members, but, as with the garbage collection metaphor, this is not without risk. This metaphor holds up pretty well until you get to the TrySubmitThreadpoolCallback function, which once again introduces a complication. Remember that the thread pool automatically closes the underlying work object that this function creates as soon as its callback executes. That means that whether or not the cleanup callback executes for this implicit work object depends on whether or not its callback has already begun to execute by the time CloseThreadpoolCleanupGroupMembers is called. The cleanup callback will only be executed for this implicit work object if its work callback is still pending and you ask CloseThreadpoolCleanupGroupMembers to cancel any pending callbacks. This is all rather unpredictable, and I therefore don’t recommend using TrySubmitThreadpoolCallback with a cleanup callback.

Finally, it’s worth mentioning that even though CloseThreadpoolCleanupGroupMembers blocks, it doesn’t waste any time. Any objects that are ready for cleanup will have their cleanup callbacks executed on the calling thread while it waits for other outstanding callbacks to complete. The features provided by cleanup groups—and, in particular, the CloseThreadpoolCleanupGroupMembers function—are invaluable for tearing down all or parts of your application efficiently and cleanly.

Kenny Kerr is a software craftsman with a passion for native Windows development. Reach him at kennykerr.ca

Thanks to the following technical expert for reviewing this article: Hari Pulapaka

Windows with C++ - Thread Pool Cancellation and Cleanup

Additional resources