Cutting Edge

Transactional Workflows

Dino Esposito

Code download available at: Cutting Edge 2007_06.exe(182 KB)

Contents

Categorizing Transactional Tasks
ACID Transactions in Workflows
Exploring the TransactionScope Activity
Configuring a Transactional Workflow for Execution
Modeling Business Processes with Windows Workflow Foundation
About Compensation
Performance and Workflow Transactions
Conclusion

Windows® Workflow Foundation, a pillar of the Microsoft® .NET Framework 3.0, provides the programming model, runtime engine, and tools for developing workflows that can be incorporated in .NET-based applications or exposed as services to any clients it may concern. So when do you really need a workflow in a custom solution?

A workflow certainly helps when the business logic and rules are subject to frequent changes. This may happen because you customize the same application to the needs of different customers or just because the customer specifically required this type of flexibility. A workflow may lie behind a business process management (BPM) server to orchestrate business services exposed to the modules of a distributed, composite application. Finally, a workflow lends itself to support the implementation of any form of long-running, cross-company business logic.

From a developer’s perspective, a workflow is a set of activities that, taken together, express the desired behavior. When called to model some real-world business logic, a workflow inevitably ends up dealing with transactional tasks. So how would you code transactional semantics in the Windows Workflow Foundation?

Nearly all enterprise systems have a workflow inside to take care of particularly complex problems, and nearly all complex problems contain a transaction. There are two main types of transactions to handle. One type includes classic, short-lived transactions that are often referred to as ACID transactions. The other type can be described as global, long-running, business-wide transactions. This sort of transaction is often a component of a business process and may consist of multiple ACID transactions. The success or failure of these ACID transactions will contribute to the overall outcome of the larger business process that is represented by the workflow.

The Windows Workflow Foundation provides ad hoc activities to make the implementation of both ACID and business-wide transactions not only effective, but relatively easy to code and update. In this column, I’ll cover the set of activities that provide for any sort of transactional tasks you might build into a Windows Workflow Foundation workflow.

Categorizing Transactional Tasks

A Windows Workflow Foundation workflow may contain one or more blocks of activities that should be seen as atomic tasks. Once you wrap such activities in an outermost transactional block, the Windows Workflow Foundation runtime ensures that all the contained activities either succeed or fail. Such transactions are referred to as ACID transactions, drawing from the description of the expected behavior—atomic, consistent, isolated, and durable (see Figure 1).

Figure 1 ACID Transaction Behavior

A whole Windows Workflow Foundation workflow may also be treated as a single long-running transaction (LRT). In this case, you use the workflow semantics to describe a business process, or a part of it, where multiple services, distinct software platforms, and different companies are involved. An LRT transaction may take minutes, days, or even weeks before the outcome is known. The process may implement a multistep operation and span over the information systems of multiple companies. Typically, an LRT transaction is made of one or more ACID transactions that succeed or fail independently. However, because the process is long and complex, it may happen that a further step in the process determines a condition that is incompatible with the result of a previous ACID transaction. In that situation, it is too late to roll back the results of an already committed transaction, but its effects must be compensated for in some way.

Although the word "transaction" is commonly used to label both ACID and long-running processes, there is a distinct difference between the two. The word transaction typically refers to a sequence of operations to be processed and treated as a single unit of work. The operation usually comes to an end in a matter of seconds and never involves a suspension or indefinitely long wait for user input or other human-led action. It doesn’t need persistence and is considered an all-or-nothing operation. As you’ll notice, the description of what constitutes a typical transaction is essentially the description of an ACID transaction.

So what exactly is a long-running transaction? Is it essentially an ACID transaction that lasts longer and can be suspended and resumed when required? A long-running transaction is a much looser use of the term transaction. An LRT is a cohesive set of actions that involve loosely coupled and independent systems.

The long-running nature of certain transactions can make managing them as an ACID transaction more complex or completely impractical. ACID transactions often require that some critical data be locked for the duration of the transaction—something that is not an issue for transactions that take only a matter of seconds. But if any of the resources involved with a particular business transaction can’t stay locked for the entire duration of the process, then the ACID pattern is not appropriate. The various operations must complete as independent operations with some surrounding logic to orchestrate actions and results, and it must be ready to compensate for errors and faults if required. The orchestrating logic can be built in a Windows Workflow Foundation workflow acting as a BPM server or constructed in a Windows Workflow Foundation composite activity that is then used in an outermost workflow.

ACID Transactions in Workflows

An ACID transaction is a short-lived transaction where the decision to commit or roll back is typically made within seconds. The rollback logic must be able to cancel the ongoing operation and all of the intermediate steps it might be made of, and it must compensate for the effect of previous operations. From the user’s perspective, it is as if nothing ever occurred. The typical database transaction coded in a stored procedure that atomically works over two or more tables is the perfect example of an ACID transaction.

An ACID transaction can be local or distributed. A local transaction involves a single resource, such as the database you’re connected to. A distributed transaction spans multiple heterogeneous resources and requires a transaction processing monitor. The Distributed Transaction Coordinator (DTC) is the transaction processing monitor for Microsoft Windows 2000 and for later releases of Windows.

To build a transactional segment in your workflow, you use the TransactionScope activity (see Figure 2). All activities composed in the scope of the transaction form a unit of work that fulfills the classic ACID schema. When all child activities have successfully completed, the transaction commits and the workflow proceeds. If an exception is thrown from any of the child tasks, the TransactionScope activity performs a rollback operation.

Figure 2** TransactionScope Activity in Visual Studio 2005 **(Click the image for a larger view)

The sample workflow shown in Figure 2 accepts an order number and proceeds with payment. Money is first withdrawn from the customer’s account and then added to the vendor’s account. The operation must be atomic in that either both steps are successfully accomplished or both fail. In any case, the involved resources must be left in a consistent state where no constraint is violated and where data integrity is guaranteed.

The internal steps are transparent to both the developer and the end user. Ideally, no external software components should access the state of any transacted resources before the state is finalized (this constraint is sometimes relaxed for performance reasons at the possible expense of correctness). Isolation, therefore, requires a serializable underlying transaction, which is the most expensive type of transaction that databases and similar resource managers support.

Exploring the TransactionScope Activity

In Windows Workflow Foundation, you use the TransactionScope activity as a container of child activities that are run sequentially and with full respect of the ACID semantics. If you use the Parallel activity within a transacted scope, then activities run concurrently within the activity, but sequentially with respect to the rest of the scope.

The TransactionScope activity exposes two properties that can be used declaratively to configure the transaction: IsolationLevel and TimeoutDuration. The IsolationLevel property is of type IsolationLevel—an enumeration defined in the System.Transactions namespace. The isolation level of a transaction determines what level of access other transactions have to internal data before a transaction is completed. Figure 3 details what happens with the most commonly used levels.

Figure 3 Isolation Levels Supported by TransactionScope

Its default value for IsolationLevel is Serializable. This is the safest and most expensive option, which is not strictly required in all cases. A serializable transaction locks data for the duration of the transaction preventing other concurrently running transactions from accessing the same data. The TransactionScope activity, however, mitigates this option by setting a timeout of 30 seconds. If you are concerned about concurrency and need to run transactions on SQL Server™ 2005, you can opt for the new Snapshot isolation level. Unlike some of the other isolation levels, Snapshot isn’t limited to being set only programmatically. It can be applied to SQL Server 2005 databases, administratively configured using the following statement:

ALTER DATABASE <name> SET ALLOW_SNAPSHOT_ISOLATION ON

The TransactionScope activity is a wrapper built around an instance of the .NET Framework 2.0 TransactionScope class—one of the most programmer-friendly classes ever designed. Just wrap everything in a TransactionScope object, and you’re pretty much finished. The object takes care of everything else. It determines whether you need a local or a distributed transaction, enlists any necessary distributed resources, and proceeds with local processing otherwise. As the code reaches a point where it won’t be running locally, it escalates to the DTC as appropriate.

You can enlist any objects with a transaction that implements the ITransaction interface. The list includes any ADO.NET 2.0 data providers, as well as Microsoft Message Queue (MSMQ).

When some code invokes the Complete method of the TransactionScope object, it indicates that all operations in the scope of the transaction are completed successfully. Note that the method does not physically terminate a distributed transaction, as the commit operation will still happen on TransactionScope disposal. However, after calling the Complete method, you can no longer use the distributed transaction.

The TransactionScope activity makes transactional tasks as easy as grouping activities in the Visual Studio® 2005 designer. Consider the code behind the activities shown in Figure 2. The key code is shown in Figure 4.

Figure 4 Code Behind the Activities

private void WithdrawFunds_ExecuteCode(object sender, EventArgs e)
{
    Console.WriteLine(“Funds withdrawn to process order: #” + 
        orderNo.ToString());

    // Execute a SQL operation on ClientAccount
    SqlHelper.ExecuteNonQuery(conn,
        System.Data.CommandType.Text,
        “use bank update ClientAccount set Balance = Balance - 100”);
}

private void AddFunds_ToVendor_ExecuteCode(object sender, EventArgs e)
{
    Console.WriteLine(“Funds added to the vendor’s account for order #” + 
        orderNo.ToString());

    // Execute a SQL operation on VendorAccount
    SqlHelper.ExecuteNonQuery(conn,
        System.Data.CommandType.Text,
        “use bank update VendorAccount set Balance = Balance + 100”);
}

The transactional code contains three Code activities. The first runs a database command and updates the balance of the client account by subtracting a given amount. Next, the same amount is added to the balance of the vendor account. Given the characteristics of the underlying TransactionScope object, databases can easily be distributed.

More importantly, the transfer of the funds must be an atomic operation. If the execution flow reaches the end of the TransactionScope activity, the transaction completes successfully and commits. Should any exception be raised at some point, the transaction automatically rolls back any work. All of this is completely transparent to the workflow developer. As a workflow developer, you don’t need to take care of compensation or even fault handling.

The sample code throws an exception if the order number is odd, as the code behind the last activity in Figure 2 shows:

void CheckConsistency_ExecuteCode(object sender, EventArgs e)
{
    if (orderNo % 2 > 0)
        throw new DiscontinuedProductException();
}

The exception thrown is a custom exception object that describes a particular situation in the application. That’s enough to trigger the rollback mechanism of the underlying .NET TransactionScope object. Figure 5 shows the application in action and the message you receive in case of a rollback. Figure 6 illustrates a successful transaction that’s been committed.

Figure 6** Money Transfer Being Committed **(Click the image for a larger view)

Figure 5** Money Transfer Being Canceled **(Click the image for a larger view)

If you need to catch any exception that originates around the workflow, and in particular around the TransactionScope activity, switch to the fault handlers view and add a FaultHandler activity. You can then configure the activity to catch a particular exception type and add to its scope as many activities as needed to handle the exception. This doesn’t make much sense in an ACID scenario, but in general you could run another transaction while handling the exception. The goal of the fault handler is to undo the partial and unsuccessful work of an activity in which an exception has occurred. However, as far as a transaction scope is concerned, the rollback is automatic and there’s no need to write ad hoc SQL commands to cancel the effects of previous database operations.

Configuring a Transactional Workflow for Execution

There are a few restrictions on the use of the TransactionScope activity that you must be aware of. First, you can’t suspend the workflow from within a transaction using the Suspend activity. In addition, a TransactionScope activity can’t be nested inside another TransactionScope activity or in any activities that implement the ICompensatableActivity interface. Examples of such activities are the CompensatableTransactionScope and CompensatableSequence activities in the toolbox.

A transactional workflow requires a persistence service; an exception is thrown if no such service is available. You register a persistence service with the workflow runtime in the client application like so:

string conn = “...”;
workflowRuntime.AddService(
    new SqlWorkflowPersistenceService(conn));

The SqlWorkflowPersistenceService class is the default persistence service and is based on a SQL Server database. Persistence occurs when an ACID transaction completes or when the workflow instance becomes idle or is programmatically unloaded. The workflow runtime engine calls methods on the persistence service to save the state of the workflow instance. The workflow runtime engine determines when and how to perform persistence. The service takes care of actually saving and loading the workflow state to and from the data store of choice.

The database used by the SqlWorkflowPersistenceService class is not created upon installation but scripts are copied on the client machine for later use. The path of the script is:

%WINDOWS%\Microsoft.NET\Framework\v3.0\Windows Workflow Foundation\SQL

You can change the default name of the database. However, you can’t change the structure of the child tables. To use a different database layout, you need a custom persistence service. At its core, a custom persistence service is a class that inherits from WorkflowPersistenceService.

Modeling Business Processes with Windows Workflow Foundation

An ACID transaction is only a small part of a real-world workflow. Multiple ACID transactions are typically combined in a single workflow to express a given business process. So it may happen that a transactional activity successfully completes, but an exception is thrown in another activity later in the workflow. In this case, no automatic rollback is possible. The rollback is possible in the context of a transaction since there’s a resource manager behind to track operations and cancel them if need be. Workflows, though, don’t support fully transactional semantics; they just support transactional activities.

When you use Windows Workflow Foundation to model a business process, you might have various transactional blocks scattered through the workflow. Windows Workflow Foundation makes available two types of transactional activities: ACID and compensatable transactions. ACID transactions are fully represented by the TransactionScope activity. They either commit or roll back and have their results persisted. But what if you need to implement transactional tasks in the context of a larger process where you can’t lock data for too long and must be able to roll back past commits? For this, you need a special type of transaction activity—the CompensatableTransactionScope activity.

CompensatableTransactionScope supports a compensation mechanism. Compensation is any logic you run at some point to undo, mitigate, or compensate for the effects of previous operations. The point is that the compensatable transaction might contain child ACID transactions that, once committed, can’t be rolled back any longer. However, in case of a further failure, their effects must be compensated for in some way. Compensation is like rollback except that the developer is called to write any code used to compensate for the work done. Not all transactional tasks have to be compensatable—for instance, tasks that can be effectively represented with ACID transactions don’t need compensation. The TransactionScope activity, in fact, doesn’t support compensation.

Figure 7presents a sample workflow that uses a couple of compensatable transaction scopes. The business process models the lifecycle of an order. Once placed, the order causes the credit card of the customer to be charged (activity named Scope_ChargeCreditCard) and then moves money into the vendor’s bank account (activity named Scope_PayOrder). Both activities are transactional and compensatable. Say an exception occurs later in the workflow because, for instance, the ordered product is no longer available. At this point, a business exception occurs and all the work done so far must now be undone. The effect of any committed ACID transactions and nontransactional sequences of activities must be compensated. Each compensatable transaction or sequence activity incorporates some logic that just serves the purpose of undoing the work—all or in part.

Figure 7** Compensatable Transactions **

In Windows Workflow Foundation, you define the undo code for each compensatable activity by switching the activity’s view to the compensation handler view (see Figure 8). In the compensation handler view, you list all activities that must be run to cancel the effect of the operation.

Figure 8** Adding Compensation Logic to Workflow Sequences **(Click the image for a larger view)

The notion of compensation is similar to the notion of rollback. However, while rollback is a purely transactional operation, compensation applies to transactional and nontransactional sequences of operations. If you compare Figures 7 and 8, you’ll notice a slightly different user interface for the two compensatable transaction scopes. In Figure 8, instead of the direct code, you see the compensation code.

About Compensation

A good question is why bother with compensation? Isn’t one big ACID transaction with automatic rollback just as good? An ACID transaction is most appropriate when operations occur within the same database or within the same information system. It is also most appropriate when operations end quickly. When different companies and services are involved, defining the process in terms of the ACID semantics is often challenging. For it to be isolated and durable, you have to keep all resources of different companies locked for the duration of the task. This is frequently unreasonable, especially if the task is long. For it to be consistent and atomic, you need ad hoc compensation code.

Another good question is who triggers the compensation code.With fault handling, you handle one or more exceptions raised by the workflow. The handler of such exceptions is another special activity—the Compensate activity (see Figure 9). The Compensate activity triggers the code that fixes constraints and business rules put at risk by a business exception. You need to bind the Compensate activity to a compensatable activity in the workflow. You do that through the TargetActivityName property in the Visual Studio 2005 Workflow Extensions designer. If you set the TargetActivityName property to the name of a particular transaction or sequence, only that transaction or sequence will be compensated. You can add multiple Compensate activities and determine the order and granularity in which compensation should occur. If you just want to run compensation logic that mimics the rollback logic of a classic ACID transaction, then you bind the Compensate activity to the whole workflow. In this case, the compensation proceeds bottom-up from the exception point to the root of the workflow.

Figure 9** Compensate Activity **

Compensation code may kick off additional ACID transactions in order to balance the effects of the workflow. Furthermore, compensation may fail. Therefore, to be on the safe side, you need to prepare appropriate exception handlers for the compensating code as well.

Performance and Workflow Transactions

One of the factors that has the deepest impact on the performance of a Windows Workflow Foundation workflow is persistence. A workflow can have a number of persistence points; some explicitly defined by programmers, others implicitly required by built-in and custom activities. The persistence service is automatically invoked when the workflow becomes idle, when the Unload method is invoked on the workflow instance, or when an activity decorated with the PersistOnClose attribute completes its work. Both built-in transactional activities defined in Windows Workflow Foundation—TransactionScope and CompensatableTransactionScope—require persistence on close. Therefore, an effective persistence service and, more importantly, an effective runtime environment for it, are critical to performance.

A transactional workflow has a couple requirements as far as its runtime machinery is concerned—persistence and transaction support. By default, the DefaultWorkflowCommitWorkBatchService service class is used to manage transactions via the .NET TransactionScope class. In this way, dynamic escalation to DTC is provided silently and only when required. However, there’s a particular scenario where you should not use the default service for the sake of performance.

When you use the out-of-the-box persistence service, you need to create a proper database—be it SQL Server 2005 or SQL Server 2000. The standard database also includes tables and stored procedures for the standard tracking service—the SqlTrackingService class. Consider a scenario where both services are enabled and share the same database, meaning you use exactly the same connection string. Persistence and tracking data are always written within the same transaction. If the database is SQL Server 2000, though, an escalation to DTC will occur unless you manage to have the services share the same connection object. Thus, when persistence and tracking data are to go in the same SQL Server 2000 database, you should use the SharedConnectionWorkflowCommitWorkBatchService instead of the default service. You add these services to the workflow runtime as shown here:

workflowRuntime.AddService(
    new SqlWorkflowPersistenceService(connString)); 
workflowRuntime.AddService(
    new SqlTrackingService(connString));
workflowRuntime.AddService(
    new SharedConnectionWorkflowCommitWorkBatchService(connString));

The SharedConnectionWorkflowCommitWorkBatchService service optimizes the workflow performance only in this particular scenario, as it avoids the overhead of extra connections to the database and DTC transactions. If your transacted workflow doesn’t need tracking or if it uses distinct databases for tracking and persistence, you are better off keeping with the default service for transaction support.

On a side note, I should point out that the tracking service and persistence service always operate within the same transaction. The tracking service has a Boolean property named IsTransactional. Setting the property to false doesn’t make the tracking service nontransactional. More simply, this means that tracking data is saved just in time when the corresponding method is called. When IsTransactional is true (the default setting), the TrackData method of the service adds data to the work batch. The work batch will then be flushed on the next persistence point in the workflow.

Conclusion

Modeling transactional tasks in Windows Workflow Foundation is actually quite easy. If you need an ACID transaction—meaning a transaction that is atomic, consistent, isolated, and durable—that will generate a response quickly, you should use the TransactionScope activity. Inside this activity, you compose other activities that access objects enlisted with the underlying ambient transaction. If, on the other hand, you need to orchestrate a number of loosely coupled services to model a task that may take a while to complete, then you’re better off using a CompensatableTransactionScope.

And remember that a transaction brings inherent costs due to required persistence and memory needs. If you only need compensation, then you should opt for the more lightweight CompensatableSequence activity.

Send your questions and comments for Dino to  cutting@microsoft.com.

Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino at cutting@microsoft.com or join the blog at weblogs.asp.net/despos.