Concurrent Affairs
Simplified APM with C#
Jeffrey Richter
Contents
Anonymous Methods and Lambda Expressions
The foreach Statement
Iterators
Asynchronous Programming
Ihave long extolled the virtues of the asynchronous programming model (APM), emphasizing that performing I/O-bound operations asynchronously is the key to producing scalable and responsive applications and components. Achieving these goals is possible because the APM allows you to use a very small number of threads to execute a large amount of work without blocking any of the threads. Unfortunately, using the APM to build a typical application or component is difficult, so many programmers just don't do it.
There are several things that make the APM difficult to implement. First, you need to avoid having state data reside on the thread's stack because data on the thread's stack cannot migrate to another thread. Avoiding stack-based data means having to avoid method arguments and local variables. Over the years, developers have come to adore arguments and local variables, as they make programming so much easier.
Second, you need to split up your code into callback methods, sometimes called continuations. For example, in one method you initiate an asynchronous read or write operation, and then you have to implement another method that will get called (possibly by a different thread) to process the result of the I/O operation. Programmers are just not used to thinking about data processing in this manner. Migrating state across methods and threads means that the state must be packaged up, which ends up complicating the implementation.
Third, there are many useful programming constructs—such as try/catch/finally statements, the lock statement, the using statement, and even loops (for, while, and foreach statements)—that can't be split up across multiple methods. Avoiding these constructs also complicates the implementation.
Finally, trying to offer features such as coordinating the result of multiple overlapping operations, cancellation, timeout, and marshaling UI control modifications to the GUI thread in a Windows® Forms or Windows Presentation Foundation (WPF) application adds even more complexity to using the APM.
In this column I will demonstrate some relatively recent additions to the C# programming language that can be used to make working with the APM significantly easier. Then I'll introduce a class of my own, called AsyncEnumerator, that builds on these C# language features in order to solve the problems I've just mentioned. My AsyncEnumerator class makes it easy and fun to use the APM in your code. With this class, you should have no reason not to use asynchronous programming, enabling your code to be scalable and responsive. Please note that the AsyncEnumerator class is part of my Power Threading library and relies on other code that is part of this library; the library is downloadable from Wintellect.com.
Anonymous Methods and Lambda Expressions
In Figure 1, the SynchronousPattern method shows how to open a file and read from it synchronously. The method is straightforward; it constructs a FileStream object, allocates a Byte[], calls Read, and then processes the returned data. The C# using statement ensures that the FileStream object is closed after processing of the data has completed.
Figure 1 Excerpts from the ApmPatterns Class
private static void SynchronousPattern() {
using(FileStream fs = new FileStream(typeof(ApmPatterns).Assembly.Location,
FileMode.Open, FileAccess.Read, FileShare.Read, 8192,
FileOptions.Asynchronous)) {
Byte[] data = new Byte[fs.Length];
Int32 bytesRead = fs.Read(data, 0, data.Length);
ProcessData(data);
}
}#region APM Pattern with Multiple Methods private sealed class ApmData {
public FileStream m_fs;
public Byte[] m_data;
}
private static void ApmPatternWithMultipleMethods() {
ApmData apmData = new ApmData();
apmData.m_fs = new FileStream(typeof(ApmPatterns).Assembly.Location,
FileMode.Open, FileAccess.Read, FileShare.Read, 8192,
FileOptions.Asynchronous);
apmData.m_data = new Byte[apmData.m_fs.Length];
apmData.m_fs.BeginRead(apmData.m_data, 0, apmData.m_data.Length,
ReadCompleted, apmData);
// Method returns after issuing async I/O request
}
private static void ReadCompleted(IAsyncResult result) {
ApmData apmData = (ApmData) result.AsyncState;
Int32 bytesRead = apmData.m_fs.EndRead(result);
ProcessData(apmData.m_data);
apmData.m_fs.Close();
// No 'using'
}#endregion private static void ApmPatternWithAnonymousMethod() {
FileStream fs = new FileStream(typeof(ApmPatterns).Assembly.Location,
FileMode.Open, FileAccess.Read, FileShare.Read, 8192,
FileOptions.Asynchronous);
Byte[] data = new Byte[fs.Length];
fs.BeginRead(data, 0, data.Length, delegate(IAsyncResult result) {
Int32 bytesRead = fs.EndRead(result);
ProcessData(data);
fs.Close();
// No 'using'
}, null);
// Method returns after issuing async I/O request
}
private static void ApmPatternWithLambdaExpression() {
FileStream fs = new FileStream(typeof(ApmPatterns).Assembly.Location,
FileMode.Open, FileAccess.Read, FileShare.Read, 8192,
FileOptions.Asynchronous);
Byte[] data = new Byte[fs.Length];
fs.BeginRead(data, 0, data.Length, result = > {
Int32 bytesRead = fs.EndRead(result);
ProcessData(data);
fs.Close();
// No 'using'
}, null);
// Method returns after issuing async I/O request
}
The ApmPatternWithMultipleMethods method shows how to perform the same operation as the SynchronousPattern method using the APM of the common language runtime (CLR). Immediately, you'll see how much more complicated the implementation is. Notice that the ApmPatternWithMultipleMethods method initiates the asynchronous I/O operation and the ReadCompleted method gets called when the operation completes. Also note that passing data between the two methods is accomplished by packaging the shared data into an instance of the ApmData class, which I had to define specifically to enable the passing of this data between these two methods. You should also notice that the C# using statement cannot be used because the FileStream is opened in one method and is then closed in a different method. To compensate for this, I had to write code to explicitly call FileStream's Close method just before the ReadCompleted method returns.
The ApmPatternWithAnonymousMethod method demonstrates how this code could be rewritten using a new C# 2.0 feature called anonymous methods, which allows you to pass code as an argument to a method. It effectively allows you to embed one method's code inside another method's code. (I describe anonymous methods in great detail in my book CLR via C#, Microsoft Press, 2006.) Notice that the ApmPatternWithAnonymousMethod method is much shorter and easier to understand—after you get used to working with anonymous methods.
First, notice that the code is simpler because it is all contained inside just one method. In this code, I am calling the BeginRead method to initiate an asynchronous I/O operation. All BeginXxx methods take for their second-to-last parameter an AsyncCallback that is a delegate referring to a method that should be called by a thread pool thread when the operation completes. Normally, when using the APM, you'd have to write a separate method, give that method a name, and pass additional data to the method via the BeginXxx method's last argument. However, the anonymous method feature allows me to just write the separate method inline so all the code to initiate the request and process the result is in context. In fact, the code looks somewhat similar to the SynchronousPattern method.
Second, notice that the ApmData class is no longer necessary; you don't have to define it, construct an instance of it, and work with any of its fields! How is this possible? Well, anonymous methods do more than just let you embed one method's code inside another method's code. When the C# compiler detects any arguments or local variables declared in the outer method that are also used in the inner method, the compiler actually defines a class for you automatically, and each variable shared between the two methods becomes a field in this compiler-defined class. Then, inside the ApmPatternWithAnonymousMethod method, the compiler generates code to construct an instance of this class, and any code that refers to the variables is compiled into code that accesses the fields of the compiler-defined class. The compiler also makes the inner method an instance method on the new class, which allows its code to access the fields easily and now the two methods can share the data.
This is an awesome feature of anonymous methods as it allows you to write code that looks like you're using method arguments and local variables, but the compiler is actually rewriting your code, taking those variables off the stack, and embedding them as fields in an object. The object can easily be passed between methods and can easily migrate from one thread to another, which is just perfect when using the APM. And since the compiler is doing all this work automatically, you can simply pass null for the last parameter to the BeginRead method since there is now no data that has to be passed explicitly between the methods and threads. I still cannot use the C# using statement, however, as there are still two different methods here even though it looks like just one method.
The following shows the output from executing the code excerpted in Figure 1:
Primary
ThreadId=1
ThreadId=1: 4D-5A-90-00-03 (SynchronousPattern)
ThreadId=3: 4D-5A-90-00-03 (ApmPatternWithMultipleMethods)
ThreadId=3: 4D-5A-90-00-03 (ApmPatternWithAnonymousMethod)
ThreadId=3: 4D-5A-90-00-03 (ApmPatternWithLambdaExpression)
ThreadId=3: 4D-5A-90-00-03 (ApmPatternWithIterator)
I had the Main method display the managed thread Id of the application's primary thread. Then I have the ProcessData method display the managed thread ID of the thread that is executing it. As you can see, the output shows that all the asynchronous patterns have the results executed by a thread other than the primary thread, while the synchronous pattern has everything executing via the application's primary thread.
I should also point out that C# 3.0 introduces a new feature called lambda expressions. The lambda expression feature is a more succinct syntax for doing the same thing as the C# anonymous method feature. In fact, this is a hassle for the C# team since it now has to document and support two different syntaxes that produce identical results. Modifying the ApmPatternWithAnonymousMethod method to use the C# 3.0 lambda expression feature gives you the ApmPatternWithLambdaExpression method shown in Figure 1. Here you see that the syntax is slightly simplified as the compiler is able to infer the type of the result argument as IAsyncResult automatically and "=>" is less typing than "delegate."
The foreach Statement
C# 2.0 introduces another feature into the C# programming language: iterators. The original intent of the iterator feature was to make it easy for developers to write code to traverse a collection's internal data structure. To understand iterators, you must first take a good look at the C# foreach statement. When you write code such as the following, the compiler turns it into what you see in Figure 2:
Figure 2 foreach Loop in C#
// Construct an object that knows how to traverse the
// internal data structure of the collection
IEnumerator < String > e = collectionObject.GetEnumerator();
try {
String s;
while (e.MoveNext()) {
// Advance to the next item
s = e.Current;
// Get the item to process
DoSomething(s);
}
} finally {
// If the IEnumerable implements IDisposable,
// clean up the object
IDisposable d = e as IDisposable;
if (d != null) d.Dispose();
}
foreach (String s in collectionObject)
DoSomething(s);
You see, the foreach statement gives you an easy way to iterate over all the items in a collection class. However, there are lots of different kinds of collection classes—arrays, lists, dictionaries, trees, linked-lists, and so on—and each uses its own internal data structure for representing the collection of items. When you use the foreach statement, you're saying that you want to iterate over all the items in the collection and you don't care what data structure the collection class uses internally to maintain its items.
The foreach statement causes the compiler to generate a call to the collection class's GetEnumerator method. This method creates an object that implements the IEnumerator<T> interface. This object knows how to traverse the collection's internal data structure. Each iteration of the while loop calls the IEnumerator<T> object's MoveNext method. This method tells the object to advance to the next item in the collection and return true if successful or false if all items have been enumerated. Internally, MoveNext might just increment an index, or it might advance to the next node of a linked-list, or it might traverse up or down through a tree. The whole point of the MoveNext method is to abstract away the collection's internal data structures from the code that is executing the foreach loop.
If MoveNext returns true, then the call to the Current property's get accessor method returns the value of the item that the MoveNext method just advanced to so that the body of the foreach statement (the code inside the try block) can process the item. When MoveNext determines that all of the collection's items have been processed, it returns false.
At this point, the while loop exits and the finally block is entered. In the finally block, a check is made to determine whether the IEnumerator<T> object implements the IDisposable interface and, if so, its Dispose method is called. For the record, the IEnumerator<T> interface derives from IDisposable, thereby requiring all IEnumerator<T> objects to implement the Dispose method. Some enumerator objects require additional resources while iterating. Perhaps the object returns lines of text from a text file. When the foreach loop exits (which could happen before iterating though all the items in the collection), then the finally block calls Dispose, allowing the IEnumerator<T> object to release these additional resources—closing the text file, for example.
Iterators
Now that you see what code the foreach statement produces, I'll take a look at how an object implements the IEnumerable<T> interface along with its MoveNext and Current members. Let's talk about arrays since they are probably the simplest of all collection data structures. The System.Array class, which is the base class of all arrays, implements the IEnumerable<T> interface. Its GetEnumerator method returns an object whose type implements the IEnumerator<T> interface. So you can easily write code to iterate through all of an array's elements:
Int32[] numbers = new Int32[] { 1, 2, 3 };
foreach (Int32 num in numbers)
Console.WriteLine(num);
As a side note, if the C# compiler detects that the collection used with foreach is an array, the compiler produces optimized code that has a simple index that advances from 0 to the length of the array minus one. But, for this discussion, let's ignore that the compiler performs this optimization and pretend that the compiler treats arrays just as it does any other collection class. In fact, you can force the compiler to treat an array just like any other collection class by casting it to IEnumerable<T> as follows:
foreach (Int32 num in (IEnumerable<Int32>) numbers)
Console.WriteLine(num);
My ApmPatterns class includes an ArrayEnumerator<T> class that knows how to return each of the items in an array in reverse order (see Figure 3). This class implements the IEnumerator<T> interface, which requires that the class also implement the non-generic IEnumerable and IDisposable interfaces as well. The AsyncEnumerator<T> class offers a public constructor that saves the passed array in a private member. There is another private member, m_index, that indicates which item from the array you are currently up to iterating; this field is initialized to the length of the array (the element past the last element in the array). You can see this in Figure 3.
Figure 3 Iterators
private static void Iterators() {
Int32[] numbers = new Int32[] {
1, 2, 3
};
// Use Array's built-in IEnumerable
foreach(Int32 num in numbers) Console.WriteLine(num);
// Use my ArrayEnumerable class
foreach(Int32 num in new ArrayEnumerable < Int32 > (numbers))
Console.WriteLine(num);
// Use my iterator
foreach(Int32 num in ArrayIterator(numbers)) Console.WriteLine(num);
}
private sealed class ArrayEnumerator < T > : IEnumerator < T > {
private T[] m_array;
private Int32 m_index;
public ArrayEnumerator(T[] array) {
m_array = array;
m_index = array.Length;
}
public Boolean MoveNext() {
if (m_index == 0) return false;
m_index--;
return true;
}
public T Current {
get {
return m_array[m_index];
}
}
Object IEnumerator.Current {
get {
return Current;
}
}
public void Dispose() {}
public void Reset() {}
}
private sealed class ArrayEnumerable < T > : IEnumerable < T > {
private T[] m_array;
public ArrayEnumerable(T[] array) {
m_array = array;
}
public IEnumerator < T > GetEnumerator() {
return new ArrayEnumerator < T > (m_array);
}
IEnumerator IEnumerable.GetEnumerator() {
return GetEnumerator();
}
}
private static IEnumerable < T > ArrayIterator < T > (T[] array) {
try {
yield
return default (T);
} finally {}
for (Int32 index = array.Length - 1; index >= 0; index--) yield
return array[index];
}
private static IEnumerator < Int32 > ApmPatternWithIterator(AsyncEnumerator ae) {
using(FileStream fs = new FileStream(typeof(ApmPatterns).Assembly.Location,
FileMode.Open, FileAccess.Read, FileShare.Read, 8192,
FileOptions.Asynchronous)) {
Byte[] data = new Byte[fs.Length];
fs.BeginRead(data, 0, data.Length, ae.End(), null);
yield
return 1;
Int32 bytesRead = fs.EndRead(ae.DequeueAsyncResult());
ProcessData(data);
}
}
All IEnumerator<T> objects are essentially state machines that keep track of which item they are up to enumerating from the collection. When the foreach loop calls MoveNext, the MoveNext method advances the state machine to the next item. In my ArrayEnumerator class, MoveNext returns false if all items have been enumerated (m_index is already at 0, the last item). If that is not the case, it decrements the m_index field and returns true. The Current property simply returns the item reflected by the current position of the state machine.
As you can see, the code is tedious to implement and I left some of the interface methods empty in order to simplify the coding. I also left out some error-checking code as well. Also, an array is a pretty simple data structure and so writing an enumerator for it is fairly straightforward. Other data structures, such as a linked list or tree, would significantly complicate the code.
One more thing that I should point out is that you cannot use my ArrayEnumerator<T> class directly with the foreach statement. In other words, this won't compile:
foreach (Int32 num in new ArrayEnumerator<Int32>(numbers))
Console.WriteLine(num);
The foreach statement expects a collection (an object whose type implements IEnumerable<T>) after the in keyword, not an IEnumerator<T>. So to use my ArrayEnumerator<T>, I had to also define a class that implements IEnumerable<T>, my ArrayEnumerator<T> class also shown in Figure 3. Now, with the ArrayEnumerable<T> class defined, I can write the following code, which will compile and execute just fine:
foreach (Int32 num in new ArrayEnumerable<Int32>(numbers))
Console.WriteLine(num);
Wow! This is quite a bit of code to write in order to iterate through a collection of items. If only there were some technology available to us that could make all of this code simpler to write. Fortunately, there is—the C# 2.0 iterator feature. An iterator allows you to write a single member that returns an ordered sequence of values. You get to write this using simple syntax to express how values should be returned and the C# compiler turns your block of code into a full-fledged class that implements the IEnumerable<T>, IEnumerable, IEnumerator<T>, IEnumerator, and IDisposable interfaces for you.
Using the C# iterator feature, I replace my AsyncEnumerable<T> class and AsyncEnumerator<T> classes with the following one member:
private static IEnumerable < T > ArrayIterator < T > (T[] array) {
for (Int32 index = array.Length - 1; index >= 0; index--) yield
return array[index];
}
And, I can invoke this member with a foreach statement as follows:
foreach (Int32 item in ArrayIterator(numbers))
Console.WriteLine(item);
As an added bonus, since this is a generic method (versus a generic class), C# type inference also kicks in and I do not have to write ArrayIterator<Int32>, making the foreach code simpler as well.
Note that, since my iterator member returns IEnumerable<T>, the compiler generates code that implements IEnumerable<T> and IEnumerable as well as IEnumerator<T>, IEnumerator, and IDisposable. However, you can write an iterator member that returns an IEnumerator<T>, in which case the compiler generates a class definition that implements only the IEnumerator<T>, IEnumerator, and IDisposable interface members.
I'll also point out that you can define an iterator member with a return type that is the non-generic IEnumerable or IEnumerator interface, then the compiler defines IEnumerable<Object>/ IEnumerable (if you specified IEnumerable as the return type), IEnumerator<Object>/IEnumerator, and IDisposable.
In an iterator, the yield return statement effectively instructs the compiler what value (collection item) to return from the method. However, an iterator doesn't actually end its execution at this point; instead, we say its execution is suspended. The next time MoveNext is called, the iterator resumes its execution with the statement immediately following the yield return statement. In addition to yield return, an iterator can contain a yield break statement that, when executed, causes the MoveNext method to return false, forcing the foreach loop to terminate. Exiting from an iterator member also causes false to be returned from the MoveNext method, forcing the foreach loop to terminate.
Asynchronous Programming
For asynchronous programming, iterators give you some really great benefits. First, within an iterator, you can write code that accesses arguments and local variables. However, the compiler is actually packing these variables up into class fields for you (just like it did for anonymous methods and lambda expressions).
Second, the compiler lets you write sequential code, but you have the ability to suspend the method midstream and continue its execution later (potentially using a different thread).
Third, you can use programming constructs such as try/catch/finally statements, lock statements, using statements, and even loops (for, while, and foreach statements) within an iterator. The compiler automatically re-writes your code so that these constructs maintain their semantics.
For the record, there are limitations associated with iterators:
- An iterator member's signature cannot contain any out or ref parameters.
- An iterator cannot contain a return statement (yield return is OK).
- An iterator may not contain any unsafe code.
- A finally block may not contain a yield return or a yield break statement.
- A yield return statement may not appear in a try block that has a catch block (yield break is OK).
- A yield return statement may not appear in a catch block (yield break is OK).
While it is good to keep these limitations in mind, for most programming scenarios, these limitations are either not an issue at all or can be worked around.
So now, armed with this knowledge of iterators, you could build an iterator that executes a series of asynchronous operations. In Figure 3, refer to the ApmPatternWithIterator member. This member is an iterator since the member's return type is an IEnumerator<T>. Notice how similar the code in this member is to the SynchronousPattern method discussed earlier. Specifically, notice how the iterator contains a using statement and how it uses an argument (ae) and local variables (fs and data).
You can tell that this member performs asynchronous operations because it calls BeginRead and EndRead (instead of just calling Read). But when using the APM, you normally call BeginRead and pass a callback method. In this case I am passing the result of calling ae.End (I'll explain this in my next column). Usually the method would return after calling BeginRead, but in an iterator I'm calling yield return, which just suspends the code.
Normally, when using the APM, the call to EndRead would be in a different method, but with iterators the call to EndRead is immediately following the yield return statement. This means that it will execute when the iterator's execution resumes. And another thread can execute this code. In fact, if you look at the output shown earlier, you see that the ApmPatternWithIterator's call to ProcessData executes via a different thread (ThreadId=3) than the application's primary thread (ThreadId=1).
If you really study this code carefully, you'll see that what's happening here is truly awesome and makes asynchronous programming almost as simple to execute as synchronous programming. But now your threads are never blocked, allowing your application to stay responsive to the user. Because threads are not blocked, there isn't a need to create more threads, so the application uses very few resources.
You can have multiple iterators executing simultaneously, which increases your application's scalability. And you can easily add the ability to cancel outstanding operations, report progress, set timers, coordinate the result of multiple overlapping operations, and have all of this execute on the GUI thread if updating the UI is desired. In addition, you can have all of this with a programming model that is very similar to the more convenient synchronous programming model.
What makes all of this actually possible is C# iterators and an AsyncEnumerator class that I've developed to drive these iterators. Hopefully I've pumped this up so much that you're dying to see my AsyncEnumerator class and how it works. Unfortunately, I've run out of space in this month's column, so you'll have to wait until my next column where I'll explain my AsyncEnumerator class in detail.
Send your questions and comments for Jeffrey to mmsync@microsoft.com.
Jeffrey Richter is a cofounder of Wintellect (www.Wintellect.com), an architecture review, consulting, and training firm. He is the author of several books, including CLR via C# (Microsoft Press, 2006). Jeffrey is also a contributing editor to MSDN Magazine and has been consulting with Microsoft since 1990.