Was this page helpful?
Your feedback about this content is important. Let us know what you think.
Additional feedback?
1500 characters remaining
Export (0) Print
Expand All

BizTalk Server 2006: BAM FAQ

Microsoft Corporation

September, 2006

Summary: This document identifies and explains common events, issues, and warnings that you may encounter when using Business Activity Monitoring (BAM). Basic BAM database functions are also discussed. The extensive use of links throughout the document enhances the depth of information covered.

When using the Business Activity Monitoring Add-In for Excel and attempting to export a BAM workbook definition to an XML document, the error message, "Invalid procedure call or argument" may occur. The cause of this error is the Excel security update (905756) released on 3/14/2006, which breaks the "Export XML" functionality of the BAM Add-In for Excel. Microsoft has since released a new security update for Excel 2003 (918419), which fixes the error caused by the previous security update (905756). After you install this new update, the BAM Add-in "Export XML" functionality works correctly.

How to Persist and Extract Data in BAM Databases

The best way to persist incoming data from a BizTalk process into BAM is to use the Tracking Profile Editor (TPE). For data coming from a non-BizTalk process, use the BAM APIs to send the data to the BAM Primary Import database.

The Tracking Profile Editor is a graphical user interface to help developers create new tracking profiles or modify existing tracking profiles for the BizTalk engine. Specifically, the TPE tool is used to drag items from orchestrations, messages, or pipelines and map them to the data items in an existing activity. An activity is deployed in the BAM Primary Import database. The TPE can drag items from orchestrations or messages and attach them to an activity. The data about individual activity instances is available for queries in a dynamically created SQL view in the BAM Primary Import database.

To extract BAM information, use a BAM view rather than going directly to the SQL tables in the Primary Import database. The view aggregates information from several tables to surface a complete view of the data. Additionally, the actual table structure may change and affect the created view in future releases.

For more information about querying BAM instance data see:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/ae4a8854-d5c2-4b36-a0ef-3f74e138306e.asp

You can find information about the BAM TPE and BAM API at these respective locations:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/064f5795-28d4-40c5-a0d2-295d3516d7f3.asp

and

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bts06CoreDocs/html/32a925f2-c7f4-4111-9c59-8865f15c6a89.asp

TPE Select Event Source Button

Clicking the TPE Select Event Source button (see the following figure) brings up the following options:

  • Select Orchestration Schedule
  • Select Messaging Payload
  • Select Context Property
  • Select Messaging Property
Aa972199.6105349a-db6c-49f4-a321-26f7689f13f0(en-US,BTS.10).gif

If you want to track anything from an orchestration, you must select the first option, Select Orchestration Schedule. The remaining three options are used only if you want to track from messaging (pipelines and ports).

For orchestration tracking, after you have selected the appropriate orchestration schedule, you can perform the mapping options as follows:

  • Milestone mapping. Select an orchestration shape in the right pane and drag the shape to the corresponding activity milestone in the left pane.
  • Non-milestone mapping. Select a shape for the context and right-click it. Click one of the following:
    • Message Payload Schemas
    • Context Property Schemas
    • Message Property Schema

Pick the appropriate schema item in the right pane and drag it to the corresponding activity item in the left pane.

The following figure shows a mapping from an orchestration schedule. By right-clicking the Receive1 shape context you can select your choice for mapping to non-milestone data.

Aa972199.b16c18eb-81e9-4839-a80d-fdeee8020d00(en-US,BTS.10).gif

Tracking Profile Editor Window

When using the TPE main window to create a tracking profile for a BAM activity that spans multiple ports and orchestrations, the following two issues may arise.

Difficulty in Specifying Ports for Data Items Tracked from Ports

When mapping data items from a message schema or context properties, the necessary ports from which to track the data must be specified. If the necessary ports are not specified, TPE will show the error message, "There are no ports associated with {0}."

The solution is to right-click the mapped data item shown in the error message, select "Set Port Mappings", and then specify the correct ports in the "Set port mappings" dialog box.

Continuation and ContinuationIDs

For the continuation to be correct, the following conditions must exist:

  • The names of the continuation and continuationID folders are identical.
  • Data from the first schedule is mapped to the continuation folder, and data from the second schedule is mapped to the continuationID folder.
  • If continuations are across ports, ports are specified for the mapped data.
  • For activities spanning multiple ports or orchestrations, a continuation is defined between successive schedules. For example, if a message flows as port -> orchestration-> orchestration->orchestration->port and everything must be tracked in one single activity, then four continuations are needed

Each of the four major BAM tracing components has trace objects to help with diagnosing problems. These trace objects are described in the following paragraphs. The major tracing components are:

  • BM.exe
  • BAM Event Bus service (also called TDDS)
  • BAM portal
  • BAM alerting

BM.exe

This executable file controls BAM tracing. You can use it in two ways:

  • At a command prompt
    Trace:on|off—Switch tracing on or off, overriding the configuration settings. This is coupled with any BM.exe command. For example:
    bm.exe deploy-all -DefinitionFile:PO.xml –Trace:On
  • In the BM.exe.config configuration file
    This file contains a section where you can enable tracing. By default, tracing is not enabled. To enable tracing, remove the comments around the following lines of code:
    <switches>
    <add name="bm" value="1" />
    <add name="Microsoft.BizTalk.Bam.Management" value="1" />
    </switches>

BAM Event Bus (TDDS)

The BAM Event Bus service is also called the Tracking Data Decode service (TDDS). To turn on tracing for the BAM Event Bus, edit the following section of the BTSNTSvc.exe.config file:

<system.diagnostics>

<switches>

<add name="Microsoft.BizTalk.Bam.EventBus" value="1" />

</switches>

<trace autoflush="true" indentsize="4">

<listeners>

<add name="Text" type="System.Diagnostics.TextWriterTraceListener" initializeData="c:\tdds.log"/>

</listeners>

</trace>

</system.diagnostics>

BAM Portal

The BAM Portal is an ASP.NET application, and follows the standard protocol for tracing. Within the web.config file, there is a section called <trace> that you can enable. For example:

<trace enabled="true" localOnly="true" pageOutput="false" />

BAM Alerting

The BAM Alerting system uses SQL Server Notification Services for the alerting infrastructure. To turn understand better how to configure SQL Server Notification Services, see the following link:

http://msdn2.microsoft.com/en-us/library/ms172486.aspx

You can subscribe to BAM events. The following table describes the events that users most commonly subscribe to

Event log Server Source Event ID Description Action

Application

TDDS

BAM EventBus Service

2

Reading error. Failed to recover from previous error

Propagate to Ops ConsoleAlert an operator Note: Critical Error (not Server Down condition)Lost the connection to the the MessageBox database, or the MessageBox database is unresponsive.

Check the MessageBox database for issues.

Application

TDDS

BAM EventBus Service

6

ExecuteEventErrorMsg.  Error(s) occurred while executing events. See TDDS_FailedTrackingData table for more details

Propagate to Ops ConsoleAlert an operator

Note: Critical Error (not Server Down condition)

What is currently deployed with the BAM dynamic infrastructure and the applications deployed are most likely not in sync. This occurs when the Activity Name is the same as one deployed or the Activity ID is not unique or parameter names are wrong.

Application

TDDS

BAM EventBus Service

5

BatchExecutionErrorMsg. TDDS failed to batch execution of streams

Propagate to Ops ConsoleAlert an operator

Note: Critical Error (not Server Down condition)Could not write to the destination database. Check to see if the destination database is running.

Application

TDDS

BAM EventBus Service

25

TDDSLockError. Either another TDDS is processing the same data or there is an orphaned session in SQL Server holding TDDS lock

Propagate to Ops ConsoleAlert an operatorNote: Critical Error (not Server Down condition)

An orphaned SQL Session is preventing the Tracking Data Decode Service (TDDS), also known as the BAM Event Bus Service, from starting.

Application

Portal/IIS

BAM Web Service

14003

Error connecting to the BAM Primary Import database.

Propagate to Ops ConsoleAlert an operator

Note: Critical Error (not Server Down condition)

This indicates the BAM Management Web service is attempting to connect with the BAM Primary Import database and is not successful.  The operator should investigate the BAM Primary Import database connectivity.

Application

Portal/IIS

BAM Event Provider

14004

Error connecting to the SQL NS

Propagate to Ops ConsoleAlert an operator

Note: Critical Error (not Server Down condition)

This indicates the BAM Alerting system (specifically, the Scheduled Aggregations portion) is attempting to connect with the BAM SQL NS database and is not successful. The operator should investigate the BAM SQL NS database for problems.

Application

Portal/IIS

BAM Aggregation Event Provider

1410

Cube DTS has not been run

Propagate to Ops ConsoleNote: Critical Error (not Server Down condition)

This indicates the Cubing DTS task has not been run. See the documentation on how to schedule the BAM Cubing DTS package.

Application

Portal/IIS

BAM EventBus Service

13023

Duplicate Activity ID

Propagate to Ops ConsoleThe Activity ID should be unique.  When an Activity ID is reused, you could end up in a state where the same Activity ID is in the Active and Completed tables.

The operator should notify the App developer of the condition and identify and remove the duplicate row.

Using the TDDS_FailedTrackingData Table

When the Tracking Data Decode service (TDDS) fails to process tracking or BAM data, it writes the failure information to the TDDS_FailedTrackingData table in binary format. This table is part of the schema of the BAM database. (This database is named BAMPrimaryImport by default, and may also be referred to as the PrimaryImport database, the PIT, or the PIDb). The HAT database is the Tracking database (referred to also as the DTADB database).

SQL Server Errors

You may receive an error message that is similar to the following in the application log of a server that is running Microsoft BizTalk Server:

Reading error. Exception information: TDDS failed to read from source database. SQLServer: SQLSERVER, Database: BizTalkMsgBoxDb.

Or in the event log you may see:

Either another TDDS is processing the same data or there is an orphaned session in SQL server holding TDDS lock. Either another TDDS is processing the same data or there is an orphaned session in SQL server holding TDDS lock. SQL Server: ServerName Database: DatabaseName

For more information about these messages, see KB article http://support.microsoft.com/?scid=kb;en-us;841334&spid=1444&sid=312

Late BAM Installation and TDDS SQL Server Connection Strings

If a BizTalk Server installation is initially done without BAM, and later BAM is installed and configured, the BizTalk Server Application service (the BizTalk Tracking Host) needs to be restarted so that its TDDS subservice can pick up the new connection string. Otherwise the Event Bus service reports an error in the Event log that a stored procedure is not found.

If you check the connection string (in the BizTalkMgmtDb database, TDDS_Destinations table, ConnectionString column where DestinationName = "Business Monitoring") you will find that it is pointing to the BAMPrimaryImport database. However, the error message indicates that the events that failed to be processed were moved to the TDDS_FailedTrackingData table in the BizTalkDTA database. This error message indicates that the connection strings encountered a timing issue during configuration, which means that the update of the original connections is still in the cache and requires recycling of the Event Bus service (the BizTalk Host under which the service is running).

To fix this problem, restart the BizTalk Server Application service. Because BAM was installed after the initial BizTalk Server installation, the connection string must be changed by the restart process. 

Using the BAM Management Utility to Administer BAM Databases

Backing Up and Restoring the BAM Analysis, Tracking Analysis, BAM Star Schema, and BAM Archive Databases

Backing Up the BAMPrimaryImport Database

After data is transferred out of the BAMPrimaryImport database into the BAMArchive database that data is no longer available to BAM. Retrieving the data would require custom work for search or historical comparison. Because of this, the BAMArchive database is not part of the BizTalk Server backup process.

The BAM Data Maintenance DTS Task works by activity. You probably want to schedule all DTS task to run serially one time per period and to write only complete newly archived tables. There is never any need to back up multiple times between data maintenance DTS tasks. You can copy whole tables to long-term storage, process data into a data warehouse, or use another method for backup

Backing Up the BAMArchive Database

This BAMArchive database is used to store old business activity data. The goal of creating a BAM Archive database is to minimize the buildup of business activity data in the BAM Primary Import database. It is best to back it up after the DTS task completes

Data Transformation Services (DTS)

Scheduling the Data Transformation Services (DTS) Package

You can create BAM views based on data stored in an online analytical processing (OLAP) cube. The Cube Update Data Transformation Services (DTS) package refreshes the data in the cube so that OLAP views reflect the correct data.

The basic idea of activity data storage is to have a separate table for each activity type, in which each record represents a different activity instance (for example, in-progress or completed).

DTS packages were renamed to SSIS packages in SQL Server 2005. These tasks are used to accomplish activities that should be executed regularly. For each activity there is one task for data management and another task for moving data to the analysis server per view.

For information about scheduling DTS packages, see:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/operations/htm/ebiz_ops_bam_admin_eqom.asp

You can find additional information at:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/bba70d81-6ddf-4f1f-a1f7-d5a5bf453bae.asp

To maximize your understanding of BizTalk Server concepts, it is useful to run the BAM tutorial. In this tutorial, you perform the steps that a business analyst completes to identify the key performance business activities that need to be tracked and aggregated. You also walk through the steps that a business analyst uses to define views for different business users.

Before you begin this tutorial, you should understand fundamental BizTalk Server concepts and introductory information about BAM. Also, before you begin Tutorial 5, you should complete all of the steps, including testing the scenario. In addition, the Order Process solution must be running successfully.

You can find the BAM tutorial here:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/ea356ca5-c387-4ea5-8c3e-e4ea2bd7a215.asp

BAM Query Web Service Documentation

There is no public documentation for the BAM Query Web service (QWS). It is not a supported interface and is merely our internal implementation of our own BAM portal. We also do not support this because we expect the format of queries to change with versions of the product and do not want to create a migration issue for our customers. We recommend that customers avoid using the QWS.

Rather, the supported and documented way of extracting instance data from BAM is to connect directly to Microsoft SQL Server™ and query the views installed by BAM. The data about individual activity instances is available for queries in a dynamically created SQL view in the BAM Primary Import database. The process is documented at this location:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/BTS06CoreDocs/html/780aa6d4-0cb4-4aee-949e-803b10dec4de.asp

Show:
© 2015 Microsoft