Measuring Maximum Sustainable Tracking Throughput
After you deploy a line of business solution onto the BizTalk Server 2006 platform, track and monitor the system to understand the following:
How the system is performing
What exceptions and errors may be occurring and why
The current state of the business processes implemented as BizTalk solutions
For many organizations, it is also important to record the actual raw messages (message bodies) flowing through the system for non-repudiation purposes. BizTalk Server 2006 provides two types of tracking functionality that address these requirements:
Data Tracking and Activity (DTA) tracking. DTA tracks a variety of user-defined message properties, orchestration debug events, message flow events, and service instance status. You use the Health and Activity Tracking (HAT) tool to query the data stored in the BizTalk DTA Tracking database (BizTalkDTADb). DTA tracking also includes the tracking of message bodies for non-repudiation and problem resolution purposes.
Business Activity Monitoring (BAM) tracking. BAM uses a user-defined tracking profile to track the state of a business process into a special set of BAM databases.
This topic describes the DTA architecture and a systematic approach to determine the maximum sustainable throughput that a system employing DTA can sustain indefinitely. Although DTA and BAM share some architectural components, this topic covers the behavior of DTA only. For information about BAM architecture, see "Business Activity Monitoring" in BizTalk Server 2006 R2 Help at http://go.microsoft.com/fwlink/?LinkId=106740.
As messages flow through the system, various tracked elements such as message bodies, properties, and events, pass through a series of processes and databases that ultimately result in writing them to the BizTalkDTADb database. After the elements are written to the BizTalkDTADb database, you can use tools, such as HAT, to query the tracked information. For information about setting up and using the BizTalkDTADb database and HAT, see "Using Health and Activity Tracking" in BizTalk Server 2006 R2 Help at http://go.microsoft.com/fwlink/?LinkId=106741.
To ensure that the system is sustainable and will run indefinitely at a given message flow rate, the pathway that tracked elements pass through on their way to the BizTalkDTADb database, and the database itself, need to remain healthy. For example, messages building up in database tables along the way, or in the DTA, can cause unbounded database file growth that is not sustainable if not properly managed.
Understanding the architecture and pathways that tracked information flows through exposes the key resources and performance indicators that you must monitor to determine how well the tracking system is keeping up with the message traffic flowing through the BizTalk Server 2006 engine.
The following figure shows an overview of the DTA tracking architecture and pathways.
Taking the numbered processes from the figure in order, all DTA tracked data flows into and out of the BizTalkDTADb database as follows:
The BizTalk Server runtime process includes a component called the interceptor. The interceptor’s job is to cache the tracked elements at runtime and, upon the next MessageBox database roundtrip (for example, en-queuing a message to the MessageBox database), forward the cached elements to the MessageBox database. The interceptor determines what elements to track by looking at the tracking configuration (also known as a tracking profile) which is obtained from the management database and cached in each host runtime instance for use by the interceptor.
As shown in the previous figure, there are two data streams inserted into the MessageBox database:
One represented by the Spool table
One by the TrackingData table
- One represented by the Spool table
The MessageBox database is the first stop for tracked data and is used to cache the tracked data so that the runtime can continue processing without being directly blocked by further tracking data processing.
To get the tracked message bodies transferred to the BizTalkDTADb database where you can view and archive them to more permanent storage, BizTalk Server 2006 employs a SQL Agent job called TrackedMessages_Copy_BizTalkMsgBoxDb which runs on each MessageBox database server. This job copies the message bodies marked for tracking to the BizTalkDTADb database.
All of the tracked data other than message bodies are moved from the MessageBox database to the BizTalkDTADb database by a service called TDDS which runs in one or more of the BizTalk Server 2006 hosts. Whenever a host is configured as Hosts Tracking on the host property pages in the BizTalk Server Administration console, the TDDS sub-service will run in every instance of that host.
Unless the BizTalkDTADb database is purged periodically, it will grow unbounded which will eventually lead to operational problems. There is a single SQL Agent job called DTA Purge and Archive (BizTalkDTADb) that performs the task of purging the BizTalkDTADb database. By default, this job runs every minute and purges all completed instances older than a user configured time (for example, 24 hours).
For more information abut the DTA Purge and Archive job, see "How to Configure the DTA Purge and Archive Job" in BizTalk Server 2006 R2 Help at http://go.microsoft.com/fwlink/?LinkID=104908.
Optionally, the DTA Purge and Archive job can also archive the BizTalkDTADb data as a SQL Backup for longer term storage and/or off-line viewing of the data. If the archived BizTalkDTADb data needs to be queried, it must first be restored as a new database in SQL Server and then you can use tools such as HAT or the SQL Query Analyzer to query it.