The following assumptions are made for this solution:

• Message order is defined at the receive ports.
  
• Messages are sent to BizTalk Server in sequential order by the upstream system.
  

This solution has the following goals:

• Maintain message order between a pair of receive and send locations even when business orchestrations process the messages in parallel.
  
• Maintain order across computer and server shutdowns.
  
• Achieve maximum sustainable throughput performance of 10 messages per second.
  

The solution does not consider:

• Messages that are sent in order by the upstream system but arrive out of order due to network latency, which are beyond the scope of this white paper.
  

The solution has the following requirements:

• A receive adapter that supports ordered reception of messages.
  
• Business orchestrations that have to propagate message ticket context properties and initialize gatekeeper correlation ID.
  

The primary function of the custom pipeline component is to stamp a sequence number as a context property in the message. The pipeline component goes out to the database table MyDB.TicketDispenser to retrieve the next available sequence that it will use to stamp the message. A database table is used because message order must be preserved across server shutdowns.

Performance is affected every time a message needs to be stamped, although this is the most straightforward way for order to be maintained during shutdowns. There are, however, ways to reduce the effect on performance. One way is to cache the sequence number in memory and periodically persist it, although this introduces timing windows where the sequence number may not be maintained across shutdowns. A more effective way is to update sequence numbers using SQL rowlock to provide the necessary atomicity for data integrity.

It is important that the ticket dispenser be placed after the pipeline disassembler in case the disassembler splits the message into one or more messages. (For example, where message 1.0 going into a disassembler comes out with messages 1.1 and 1.2.)

The acquisition of the sequence number is done in the Execute method of the pipeline component. As such, it is necessary to participate in the pipeline component’s transaction for the thread of execution. Stored procedures are used to perform the database updates called from the Execute method as follows:

CREATE PROCEDURE UpdateSequence
@ReceiveLocationName as nvarchar(50)
AS
BEGIN
    -- SET NOCOUNT ON added to prevent extra result sets from
    -- interfering with SELECT statements.
    SET NOCOUNT ON;

    declare @val int
    declare @lastSeq int
    declare @msgInQue int
                
    UPDATE TicketDispenser set SequenceNumber =  SequenceNumber + 1
       FROM TicketDispenser WITH (ROWLOCK)
       WHERE ReceiveLocationName = @ReceiveLocationName
 
    if @@ROWCOUNT=0
    BEGIN
      SET @val=1
      SET @lastSeq=0
      SET @msgInQue=0
      INSERT into TicketDispenser(ReceiveLocationName, SequenceNumber, LastSequenceSent, MsgInQueue) values(@ReceiveLocationName, @val, @lastSeq, @msgInQue)
    END

    SELECT @val=SequenceNumber from TicketDispenser with (ROWLOCK) WHERE ReceiveLocationName = @ReceiveLocationName

    SELECT @val
END
GO

MyDB.TicketDispenser

CREATE TABLE [TicketDispenser] (
[SequenceNumber] [int] NULL ,
[LastSequenceSent] [int] NULL ,
[MsgInQueue] [int] NULL ,
[ReceiveLocationName] [nvarchar] (50) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,
CONSTRAINT [PK_TicketDispenser] PRIMARY KEY  CLUSTERED 
(
[ReceiveLocationName]
)  ON [PRIMARY] 
) ON [PRIMARY]
GO

The function of the gatekeeper orchestration is to resequence messages according to the order that they arrive to ensure they are sent out in the same order. The basic algorithm is to queue out-of-order messages that are not ready to be sent out. When a message is received its sequence number is checked to see if it is equal to the last sequence number sent plus one. The message is sent out if this is the case; otherwise the message is queued. The sequence number of queued messages is maintained in memory. This is advantageous because orchestration state is preserved across orchestrations during dehydration and rehydration. Thus, message order is maintained during system restart without maintaining a separate database by leveraging orchestration state management.

The gatekeeper orchestration subscribes to the correlation initialized by the business orchestrations so that the MessageBox database publishes all post-processed messages to it. Figure 2 shows an activating receive shape (FirstBizMsg) where the OrderedDelivery.PropertySchema.Destination correlation field is initialized by the activating message. The other receive shapes will follow on this correlation to receive all messages that have been post-processed by the business orchestrations.

Figure 2. Gatekeeper First Message

A .NET WaitQueue class implements a sorted list to store XLANG/s message references (excluding the body payload) that is not yet ready to be sent. It is important that this class perform the necessary XLANG/s message lifetime management of reference counting to keep track of whether there is any object referencing a particular message. Normally the XLANG/s runtime performs message reference counting for messages that are part of the orchestration and that are stored in the MessageBox database. In this case, the message is being referenced in .NET code outside of the runtime.

The .NET WaitQueue class method RemoveMessage() is called when a message is found in the queue that is the next sequenced message to be sent out. The RemoveMessage() calls the dispose method of the message to signal to the runtime that this instance is done with the message.

Figure 3 shows the main loop of the gatekeeper orchestration that receives and processes all messages after the first activating message. There are two branches to this loop. One branch checks for message order before the message is sent out and the other branch handles the control message to shut down this singleton.

Figure 3. Gatekeeper Main Loop

Message order is determined by inspecting the stamped sequence number of each message against the sequence number of the last message sent. There is additional logic to check the WaitQueue every time a message is sent. In other words, every time a message is sent out it checks the WaitQueue to see if the next sequence number is queued. Since this is a sorted list it allows potentially for a batch of sequential messages to all be sent one after another. For example, if the next message to be sent in the sequence is 55, while messages 56 through 100 are queued up, when message 55 eventually arrives and is sent out, the logic path will ensure that messages 56 through 100 are sent out in quick succession to flush the queue.

This gatekeeper orchestration is able to respond to control messages to shut down. Upon receiving the message to shut down it checks to see if there are messages in the WaitQueue. If there are none it writes out the last sequence sent before it shuts down. This value is used to initialize the next sequence number when the gatekeeper is reactivated so that order can be maintained across shutdowns of the singleton gatekeeper. If there are messages in the WaitQueue the gatekeeper suspends and waits for user intervention.

We will consider a couple of error conditions and possible ways to handle them.

• Hole in the sequence message: This is a scenario where the gatekeeper is waiting for a message with a specific sequence number and the message never arrives. This can happen if a business process consumes one of the messages and does not forward it. There is no easy algorithmic solution to this problem. Depending on the application, a practical approach is to skip the message after a designated time-out value, and to log the event. To accommodate for known latency a retry count can be used in conjunction with the time-out value before a message is skipped.
  
• Messages held in the gatekeeper orchestration when it is shutting down: Since XLANG/s messages have to exist within the context of an orchestration, routing messages to a database does not work because the message lifetime context is lost and the runtime does not provide a mechanism to load stored messages. It is possible to route messages to a surrogate orchestration while the gatekeeper restores itself (where it was perhaps shut down due to memory pressure), and to receive the messages back when it comes back up. The restored gatekeeper also reloads the WaitQueue together with the rest of the variable states like next sequence number. This solution, however, is fairly complicated to use as a general solution to handle all such situations.
  

We will consider some tradeoffs for the design and implementation described in this white paper.

• Why is the gatekeeper implemented as an orchestration instead of a send custom pipeline component?
  One clear reason against a pipeline implementation is a deadlock that can occur when the pipeline instance gets throttled while BizTalk Server is waiting for a message with the next allowed sequence. Once throttled even when the specific message finally arrives, BizTalk Server does not give the thread of execution to the pipeline because it has been throttled out already. This happens when BizTalk Server is under high memory pressure and the message queue backlog is high.
  By contrast, when you use an orchestration you can leverage the built-in memory management mechanisms in dehydration and rehydration to manage state. You also do not have to maintain a separate persistent store required by a pipeline component for system recovery. Furthermore, orchestrations provide flexibility to model business semantics in the orchestration business logic. An example is correlating messages and patient IDs.
  
• Database configuration
  The custom pipeline component uses a database that is configured in the BizTalk Server btsntsvc.exe.config file. Optionally, the SSO database can be used to allow central configuration when deploying to a server farm. This enhances security with an encrypted connect string and password.
  
• Content-based ordering (CBO)
  The ticket dispenser custom pipeline component may be implemented with an orchestration doing in-memory sequence stamping. There is no need to maintain a user database for sequence number persistence, as this is taken care of by orchestration state management in the runtime.
  
• Gatekeeper
  The advantages of implementing the resequencer (gatekeeper) as an orchestration are that it increases performance due to no database cost, and because of the ability to leverage the orchestration correlation feature for routing.
  
• Ticket Dispenser
  The disadvantages of implementing the ticket dispenser as an orchestration as opposed to a custom pipeline component, as described in this paper, is that it increases known singleton state bloat and management issues.
  

The following considerations explain more advanced implementations of a gatekeeper orchestration to further improve ordered delivery for specific business requirements.

• Multiple Receive Locations
  The sample in this white paper demonstrates ordered delivery between a pair of receive and send ports. However, it is common to have more than one pair of receive and send ports. As previously discussed, it is possible to support multiple pairs of ports in the TicketDispenser table, where each row represents a pair of ports. To fully support this feature there needs to be a gatekeeper singleton for each pair of ports. There is an elegant way to achieve this by making the correlation value of each singleton the receive location name.
  
• Relative Ordering
  It can be argued that businesses need relative ordering of all messages pertaining to a single entity such as a patient ID more than absolute ordering of all messages going through the system for all patients.
  This can be easily done by activating a singleton gatekeeper for each patient and correlating messages based on patient ID. The added benefit of the WaitQueue is smaller since it is only dealing with messages for one patient. The lifetime of the orchestration is defined by the patient check-in that activates the singleton and patient check-out which ends the orchestration. This pattern is suitable for other applications like stock trades, processing a buy or sell order through order confirmation, selling or buying orders, funds settlement, and closing the order.
  
• Sequence Wraparound
  The size of the sequence number in this implementation is an integer. The sequence number will at some point wrap around back to one. This implementation assumes the custom pipeline and ticket dispenser orchestration will be recycled before reaching the integer size boundary before wraparound occurs.
  A more robust design is to add a batch field to the context property schema which contains a GUID. The ticket dispenser acquires a batch GUID and stamps messages with sequence numbers within the integer range. When the maximum integer is reached a new GUID is acquired to denote a new batch and the sequence number starts at one. Consequently, the gatekeeper orchestration has to compute message sequence numbers within the context of its batch.
  In the custom pipeline, when the maximum size in the integer range is reached, a new batch GUID is generated to be used in the batch field and the sequence number begins with one. It is not necessary to synchronize sequence numbers between the ticket dispenser and gatekeeper in this scheme. The custom pipeline stamps batch and sequence numbers and the gatekeeper orchestration processes the next sequence based on the sequence number in each batch.
  

This section contains information about performance and system configuration for the solution. For background information about BizTalk Server performance, see http://go.microsoft.com/fwlink/?LinkId=93137.

Maximum sustainable throughput

Hardware configurations

Message size

Software configuration

MSMQ adapter configuration

BizTalk Server configuration

Multicomputer Configuration (five computers)

<?xml version="1.0" encoding="utf-16" ?> 
  <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://OrderedDelivery.PropertySchema.PropertySchema" targetNamespace="http://OrderedDelivery.PropertySchema.PropertySchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
  <xs:appinfo>
  <b:schemaInfo schema_type="property" xmlns:b="http://schemas.microsoft.com/BizTalk/2003" /> 
  </xs:appinfo>
     </xs:annotation>
         <xs:element name="Destination" type="xs:string">
   <xs:annotation>
  <xs:appinfo>
  <b:fieldInfo propertyGuid="4be0511f-ed6d-4f15-a5cd-5f52f6c3f2f9" propSchFieldBase="MessageContextPropertyBase" /> 
  </xs:appinfo>
  </xs:annotation>
   </xs:element>
 <xs:element name="GateKeeperStop" type="xs:boolean">
<xs:annotation>
<xs:appinfo>
  <b:fieldInfo propertyGuid="46e85930-7176-4439-a6ec-e9794e6b923c" propSchFieldBase="MessageContextPropertyBase" /> 
  </xs:appinfo>
  </xs:annotation>
  </xs:element>
   <xs:element name="MessageId" type="xs:int">
  <xs:annotation>
  <xs:appinfo>
  <b:fieldInfo propertyGuid="72440ecc-6734-44a8-ba8d-a5c45bff3e7b" /> 
  </xs:appinfo>
  </xs:annotation>
  </xs:element>
   <xs:element name="TestDriverStart" type="xs:boolean">
  <xs:annotation>
  <xs:appinfo>
  <b:fieldInfo propertyGuid="6b0a87e0-0db2-4c36-aae8-bcb2415e37f8" /> 
  </xs:appinfo>
  </xs:annotation>
  </xs:element>
   <xs:element name="MessageType" type="xs:string">
  <xs:annotation>
  <xs:appinfo>
  <b:fieldInfo propertyGuid="8b13396a-6a6c-4bd3-a860-97b52fa05da9" propSchFieldBase="MessageContextPropertyBase" /> 
  </xs:appinfo>
  </xs:annotation>
  </xs:element>
  </xs:schema>

Figure 4. Test Driver

The receive locations for each of the business orchestrations are configured with the custom pipeline component in order to stamp a sequence number.

Note
PatientId and Symptoms are business elements to simulate a healthcare application, and are not used in the solution to determine order.

<ns0:BusinessMessage xmlns:ns0="http://OrderedDelivery.BusinessMessage">
  <PatientId>10</PatientId>
  <Symptoms>Symptoms_0</Symptoms>
  <MessageId>10</MessageId>
  <GateKeeperStop>false</GateKeeperStop>
  <TestDriverStart>false</TestDriverStart>
  <MessageType>ADT1</MessageType>
</ns0:BusinessMessage>

The business orchestrations propagate the context properties and initialize the message correlation to “Gatekeeper” so that the message will be published to the gatekeeper for resequencing.

Figure 5. Business Orchestration

Use the following procedure to set up the solution.

 To set up the solution

1. Unzip the solution to your local computer, and place the sample solution in the root directory of C:\.

2. Create the following directories:

   • C:\Input
     
   • C:\Output
     

Use the following solution to build and deploy the solution.

 To build and deploy the solution

1. Create a strong name assembly file called Sequence.snk and place it in the sample root directory C:\OrderedDelivery.

   You can do this at a Microsoft Visual Studio® command prompt: sn.exe –k Sequence.snk.

2. Using Visual Studio 2005, open OrderedDelivery.sln.

3. Build the Helper project:

   1. Right-click the Helper project to add C:\OrderedDelivery\Sequence.snk to the project.
      
   2. Specify in the project properties that Sequence.snk is used as the strong name key file to sign the assembly.
      
   3. Build Helper project which will add Microsoft.Samples.BizTalk.OrderedDelivery.Helper.dll to the global assembly cache (GAC).
      

4. Build the TicketDispenser project:

   1. Right-click the TicketDispenser project to add C:\OrderedDelivery\Sequence.snk to the project.
      
   2. Specify in the project properties that Sequence.snk is used as the strong name key file to sign the assembly.
      
   3. Build TicketDispenser project which will add Microsoft.Samples.BizTalk.OrderedDelivery.Pipeline.CustomComponent.TicketDispenser.dll to the global assembly cache (GAC).
      

5. Build the Schemas project:

   1. Add a reference and browse to C:\Program Files\Microsoft BizTalk Server 2006 to add Microsoft.BizTalk.DefaultPipelines.dll to the project.
      
   2. Build the Schemas project (right-click the project and then select Build).
      

6. Build the Pipeline project:

   1. Open TicketDispenserReceivePipeline.btp.
      
   2. In the Toolbox pane, right-click BizTalk Pipeline Components, and then select Choose Items.
      
   3. Click the BizTalk Pipeline Components tab and browse to C:\OrderedDelivery\TicketDispenser\bin\release (or debug), and then select Microsoft.Samples.BizTalk.OrderedDelivery.Pipeline.CustomComponent.TicketDispenser.dll. Select the check box so that it will be displayed.
      
   4. Drag the Ticket Dispenser Component to the Validate stage in the Pipeline designer.
      
   5. Build the Pipeline project (right-click the project and then select Build).
      

7. Build the Business project (right-click the project and then select Build).

8. Build the GateKeeper project (right-click the project and then select Build).

9. Build the TestDriver project (right-click the project and then select Build).

10. Ensure that the BizTalk projects deployment configuration database property matches the actual BizTalk group configuration database (shown in the BizTalk Server Administration Console).

11. Deploy the Schemas project (right-click the project, and then select Deploy). View the following schemas in the BizTalk Server Administration Console Schema node:

    • OrderedDelivery.TicketDispenserPropertySchema
      
    • OrderedDelivery.PropertySchema
      
    • OrderedDelivery.BusinessMessage
      

12. Deploy the Pipeline project (right-click the project, and then select Deploy). View the pipeline component in the BizTalk Administration Console Pipelines node:

    • Pipeline.TicketDispenserReceivePipeline
      

13. Deploy the Business project (right-click the project, and then select Deploy). View the orchestrations in the BizTalk Server Administration Console Orchestrations node:

    • OrderedDelivery.MyBiz
      
    • OrderedDelivery.MyBiz2
      
    • OrderedDelivery.MyBiz3
      

14. Deploy the Gatekeeper project (right-click the project, and then select Deploy). View the orchestration in the BizTalk Server Administration console Orchestrations node:

    • OrderedDelivery.Gatekeeper
      

15. Deploy TestDriver project (right-click the project, and then select Deploy). View the orchestration in the BizTalk Server Administration console Orchestrations node:

    • OrderedDelivery.TestDriver
      

Use the following procedure to configure the database.

 To configure the database

1. Create MyDB database in SQL Server.

2. Create TicketDispenser table in MyDB using CreateTicketDispenser.sql.

3. Import stored procedures defined in Sequence.sql into MyDB.

Use the following procedure to configure BizTalk Server.

 To configure BizTalk Server

1. Add the following to the BizTalk configuration file btsntsvc.exe.config:

   <appSettings>
   <add key="OD_ConnectString" value="Data Source=localhost;Integrated Security=SSPI;Initial Catalog=MyDB;Enlist=false;" />
   <add key="OD_ReceiveLocation" value="ReceiveLocation1"/>
   <add key="OD_Biz1_Message" value="C:\OrderedDelivery\Data\BusinessMessage_1.xml"/>
   <add key="OD_Biz2_Message" value="C:\OrderedDelivery\Data\BusinessMessage_2.xml"/>
   <add key="OD_Biz3_Message" value="C:\OrderedDelivery\Data\BusinessMessage_3.xml"/>
   </appSettings>

2. Restart the BTSNTSvc service.

3. Open the BizTalk Server Administration console and navigate to BizTalk Application 1 where the solution was deployed.

4. Right-click BizTalk Application 1 and select Configure to create the following ports and locations:

   Create TestDriver Receive port

   1. Right-click Receive ports to create a One-Way Receive port naming it TestDriverReceivePort.
      
   2. Click Receive Locations, and then click New to specify:
      Name: TestDriverReceiveLocation
      Transport Type: File
      Receive Folder: C:\input
      File mask: *.xml
      Receive pipeline: XMLReceive
      

   Create the MSMQT adapter

   1. Right-click Adapter in Platform Settings.
      
   2. Name the adapter MSMQT.
      
   3. Choose MSMQT from the adapter drop-down list.
      
   4. Stop the MSMQ service through the Device Manager MMC to ensure that messages are consumed only by MSMQT.
      
   5. Restart the BizTalk host instance for MSMQT configuration to take effect.
      

   Create MSMQT receive ports and locations

   1. Right-click Receive ports to create a One-Way Receive port naming it BusinessReceiveMSMQTPort.
      
   2. Right-click Receive Locations, and then click New to specify the following:
      Name: BusinessReceiveMSMQTPortLocation
      Transport Type: MSMQT
      Configure Queue name: inQueue
      Receive pipeline: TicketDispenserReceivePipeline
      

   Create the MSMQT send port for the TestDriver

   • Right-click Send port to create a new Static One-way Send Port, and then specify the following:
     Port Name: TestDriverMSMQTSendPort
     Transport: MSMQT
     Configure Destination Queue: DIRECT=OS:localhost\Private$\inQueue
     Send pipeline: XMLTransmit
     

   Create a send port for the gatekeeper

   • Right-click Send port to create a new Static One-way Send Port and specify the following:
     Port Name: GatekeeperSendPort
     Transport: File
     URI: C:\output
     File name: %messageID%\*.xml
     Send pipeline: XMLTransmit
     
   • Click Transport Advanced Options and select Ordered Delivery and Stop sending subsequent messages on current message failure.
     

   Bind orchestrations MyBiz, MyBiz1, MyBiz2

   • Host to BizTalkServerApplication
     
   • ReceiveMsg to BusinesReceiveMSMQTPort
     

   Bind Gatekeeper orchestration

   • Host to BizTalkServerApplication
     
   • GatekeeperSendMsg to GatekeeperSendPort
     

   Bind TestDriver orchestration

   • Host to BizTalkServerApplication
     
   • Port_StartTest to TestDriverReceivePort
     
   • Port_MSMQT to TestDriverMSMQTSendPort
     

Use the following procedure to run the solution.

 To run the solution

1. Start the BizTalk application using the BizTalk Server Administration console.

2. Attach another instance of Visual Studio to the btsntsvc.exe process to see trace output.

3. Drop StartTestDriver.xml into C:\Input.

4. You should see a max of 15 instances of biz.odx and one gatekeeper orchestration.

5. You should see 15 files in C:\output directory when run is completed.

6. Verify order in the trace output.

7. Verify the TicketDispenser table shows sequence number of 15 in the SQL Query Analyzer by typing select * from TicketDispenser.

Use the following procedure to stop the gatekeeper orchestration.

 To stop the gatekeeper orchestration

1. Drop StopGateKeeper.xml into C:\Input.

2. Verify that the BizTalk Server hub view shows there is no gatekeeper orchestration instance.

Use the following procedure to verify order durability.

 To verify order durability

• To see that order is maintained across server shutdown and from one instance of the gatekeeper to another, rerun the test repeating “Running the solution”. You can also power cycle the computer during a run.

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred

© 2007 Microsoft Corporation. All rights reserved.

Microsoft, BizTalk, Visual Studio, and Windows Server are trademarks of the Microsoft group of companies.

All other trademarks are property of their respective owners.