General Troubleshooting

Troubleshooting begins with the BizTalk Server Administration console and the Health and Activity Tracking (HAT) tool, and can extend into a number of different tools and utilities. You may find yourself going from the Orchestration Debugger in HAT to the Network Monitor application to look at packets containing information relevant to your solution. This section discusses the many different tools and utilities available along with key functionality that may prove useful when performing basic troubleshooting.

The BizTalk Server Administration console will likely be your tool of choice when first confronted with an issue because it provides a comprehensive view of currently running processes and messages, suspended messages, routing error messages, and other information from active BizTalk processes.

The Administration console enables developers and system administrators to:

  • Quickly view work in progress and suspended items (with different groupings).
  • View message details including general properties, context information, and message parts.
  • View message flow.
  • Save a message to a file for investigation using non-Microsoft tools.

If you want to track rule-firing events and debug orchestrations, you must use the HAT tool.

Viewing In-Process and Suspended Items

The Group Hub page provides a quick way to access in-process, suspended, and other message instances. Using the Group Hub page, you can quickly view dehydrated orchestrations, retrying and idle ports, ready service instances, scheduled service instances, resumable and non-resumable service instances, and suspended MSMQT messages. You can also see instances grouped by application, service name, error code, and URI.

To view Suspended (Resumable) messages
  1. Click Start, point to Programs, point to Microsoft BizTalk Server 2006, and then click BizTalk Server Administration.

  2. In the left pane, expand Console Root and then expand BizTalk Server 2006 Administration.

  3. Click BizTalk Group [MyServer:BizTalkMgmtDb].

  4. In the Group Overview pane, press F5, or on the Action menu, click Refresh to update the statistics in the Group Overview pane.

  5. In the Group Overview pane, under Suspended Items, click Resumable to view Suspended (Resumable) instances. Click other highlighted labels to view related instances.

Viewing Message Flow

By using the Message Flow view, you can trace the path of an activation message through your business processes. To view instance message flow, right-click a service instance in the Query Results pane of any service instance search and then click Message Flow.

If a message flow is not applicable to the instance, the option is not available.

As the following screenshot illustrates, message flow provides information about the different events that happened during the flow of an instance, including direction (in/out), status, date and time, message instance, and related details.

Message flow for sample message

Saving a Message to a File

You can save messages to a file for further investigation using your preferred tools. You can save message information to a file from the Messages tab of the Service Details tab in the Administration console.

To save a message to a file using the Administration console
  1. Click Start, point to Programs, point to Microsoft BizTalk Server 2006, and then click BizTalk Server Administration.

  2. In the left pane, expand Console Root and then expand BizTalk Server 2006 Administration.

  3. Click BizTalk Group [MyServer:BizTalkMgmtDb].

  4. In the Group Overview pane, press F5, or on the Action menu, click Refresh to update the statistics in the Group Overview pane.

  5. In the Group Overview pane, under Suspended Items, click Resumable (or another highlighted label with at least one instance) to view Suspended (Resumable) instances.

  6. In the Query Results pane, right click an instance, and then click Service Details.

  7. On the Service Details screen, click the Messages tab, right-click a message, and then click Save To File. After you have read the message about confidential data, click OK..

    Do not save the message if you are unsure about file security or whether the message contains confidential data.

  8. Choose a directory for instance-related files, and then click OK. If the files are saved successfully, you will receive a success message.

  9. Use Windows Explorer to navigate to the directory you specified in the preceding step, and view the data file ({partname}.out) and context file ({messagename}.xml) for the selected message instance.

When you configure BizTalk Server 2006 using a custom configuration, consider the following:

  • The account you are logged on as must be part of the local administrators group and have System Administrator rights on SQL Server.
  • The default accounts generated by BizTalk Server and listed in the custom configuration manager are local groups. In a multiserver environment you must substitute the local groups with domain groups.
  • The account you are logged on must be a member of the OLAP Administrators group on the OLAP computer if configuring.
    If you change the group membership of the current logged-in user, and the group is also a member of the domain, you should logout and login after changes are completed. Failure to do this will result in denied access because new group membership will not be reflected by the current login.

The Health and Activity Tracking tool enables developers and system administrators to:

  • View tracked message events, properties, and bodies
  • Debug orchestrations and replay the execution of a specific orchestration instance using the Orchestration Debugger.
  • Track rule-firing events.
  • Retrieve messages by specifying criteria such as message schema and message property/value pairs.
  • Search for archived data in an archived or backed-up Tracking database.

If you are looking for configuration information, instance status, message contents, and other instance information, you can use the BizTalk Server Administration console.

Viewing Tracked Message Events, Properties, and Bodies

You can use the Health and Activity Tracking tool to view tracked message events, properties, and message bodies including multipart messages.

For troubleshooting in a development, staging, or production environment, you should enable all tracking options so that you can see artifact events, message properties, message bodies, and orchestration events in detail. This provides a rich set of tracked information that you can use to re-create the sequence of events in your system and debug your application.

You can use either the Find Message view or one of the predefined queries to find messages. Use the Find Message view to build advanced queries using message properties, message events, message body, schema information, strong name, and any of the message's promoted properties. The predefined queries provide common queries; after you run them you can modify them and save them for reuse.

To run a predefined query
  1. Click Start, point to Programs, point to Microsoft BizTalk Server 2006, and then click Health and Activity Tracking.

  2. On the HAT Tools menu, click Queries, and then choose Most recent 100 service instances or one of the other query choices as appropriate to your situation.

    The results will be displayed in the Query Builder view and will include tracked messages. For example, the following screenshot shows three tracking message events and tracks the receipt of a message through a receive port, processing through an orchestration (HelloWorld sample), and transmission through a send port.

    100 most recent service instances
  3. Right-click an individual entry to view the message flow, save the message body, or perform other actions relevant to the type of entry. For example, you cannot save a service instance to disk, but you can save related messages to disk.

To run a custom query using the Find Message view
  1. Click Start, point to Programs, point to Microsoft BizTalk Server 2006, and then click Health and Activity Tracking.

  2. On the HAT Tools menu, click Reporting, and then choose Find Message.

    Custom query in find message view
  3. In the Find Message view, click Schema to open the Schema Selection screen, and then click one of the schemas. This limits searches to messages whose schema matches the selected schema.

  4. Change the From and To dates as appropriate to your situation. To change, click the drop-down arrow and use the calendar to select a different date. The system defaults to an eight-day range from a week before the current date to 24 hours after.

  5. Enter optional parameters for Port, Party, and the message properties filter. There is an implicit AND clause between all conditions, so messages must match all of the search criteria to be considered a match.

    Message properties filter values are available only if the property has been previously configured for tracking.

  6. Click Run Query. Matching items are displayed below the search criteria.

You can also save tracked message bodies to a file and view them using non-Microsoft tools.

To save tracked message bodies
  1. Click Start, point to Programs, point to Microsoft BizTalk Server 2006, and then click Health and Activity Tracking.

  2. On the HAT Tools menu, click Queries, and then choose Most recent 100 service instances or one of the other query choices as appropriate to your situation.

  3. Right-click a service item that has been configured to track messages, and then click Save All Tracked Messages.

  4. If it has not been turned off, you will be cautioned that the tracked messages may contain confidential data. After you have read the message, click OK.

    Save tracked messages containing confidential information in a secure location. If you do not, users without proper authorization may be able to view the confidential information.

  5. Choose a destination folder. Make sure the folder is secure if your messages contain confidential information. Click OK to continue.

    All of the tracked messages related to this service will be saved to the selected destination folder unless there was a problem or no messages were found. On success you will be shown a list of service instance IDs and associated message IDs that were saved, as in the following figure:

    Screen showing saved messages status

    If a message could not be saved, the Save messages status column will contain a description of the possible error(s), similar to the following:

    "The message was not found in the Message Box or the Tracking database. This may be caused by the following conditions: (1) message tracking is not enabled; (2) the message(s) is no longer referenced by a running or suspended service instance; (3) the Message Box tracking tables have been automatically purged; or (4) the SQL Server agent is not running on the Message Box servers."

    To repair, first ensure that message tracking is enabled and the SQL Server agent is running on the MessageBox servers. After you confirm this, verify that the tracking tables have not been purged by running a custom query with dates that include the days your tracked messages were created. For messages associated with a service instance that are not tracked, use the Administration console group hub page to discover whether the message(s) are referenced by a running or suspended service instance. If they are not, you will not be able to save them.

    The tracked messages and related files are now saved in the destination folder you selected.

BizTalk Server does not track messages that do not successfully make it through one of the tracking points. In some cases—such as when a message is suspended because it is invalid, or if no host is expecting the message—it may be placed in the Suspended queue without being tracked. If you terminate this message there will be no record of it.

Knowing which tracking points were reached and the first tracking point that was missed will shed light on where in the process the message is failing. For example, if a message fails to hit a tracking point on a receive adapter, the system generating the message may be failing to generate a message or a message with the expected schema, or a custom component within the receive pipeline may be throwing an exception prior to message identification. Your debugging strategy will then be based on the stage (and component) that you suspect caused the error.

You can use the BizTalk Server Administration console to enable message body and message property tracking.

Debugging an Orchestration Instance

Unlike traditional Visual Studio development projects that can be debugged directly within the development environment, a BizTalk orchestration can be debugged only after it has been deployed and executed on a BizTalk Server computer using the Health and Activity Tracking tool or the Administration console. Orchestrations can be debugged using both Replay mode within HAT and Live mode within HAT or the Administration console.

When using Replay mode, you can track the activity of an orchestration on a shape-by-shape basis to understand how a message was processed. Orchestration activity is listed in tabular format and you cannot set breakpoints or view individual variables and other values.

With Live mode you can set breakpoints at various locations within an orchestration. Setting breakpoints at the class and instance level results in a more interactive debugging experience.

For more information about debugging orchestrations, see Troubleshooting Orchestrations.

Reconstructing Rule Events

When appropriate, the Message Flow view provides a link to access details about tracked rule events. To use this feature, you must use the Administration console to configure tracking for a policy.

To configure tracking for a policy
  1. Click Start, point to Programs, point to Microsoft BizTalk Server 2006, and then click BizTalk Server Administration.

    To perform the procedure in this topic, you must be logged on with an account that is a member of the BizTalk Server Administrators group.

  2. In the console tree, expand the BizTalk group and the BizTalk application for which you want to configure tracking for a policy.

  3. Click Policies, right-click the policy, click Properties, and then click Tracking.

  4. Select the tracking options you want as described in the following table.

    Use this To do this

    Fast activity

    Select this check box to track the instance data on which the policy operates.

    Condition evaluation

    Select this check box to track the true/false results of conditions in the selected policy.

    Rule firings

    Select this check box to track the actions started as a result of the policy.

    Agenda updates

    Select this check box to track updates to the agenda. The agenda contains a list of actions that are "true" and need to fire.

  5. Click OK.

Searching Archived Data

The HAT tool defaults to search live data available in the BizTalkDTADb database. You can search archived data by enabling archived data in HAT Preferences and specifying the archive database server and database name.

The Windows event logs contain information, warnings, errors, and other messages relevant to BizTalk Server and other applications running on your BizTalk Server computer. You will find the Application, Security, and System event logs useful for diagnosing application and server problems, and you can also write your own custom events into one of the existing event logs (such as Application) or into a custom event log specific to your application needs.

Accessing Event Logs

You can access and view events in the Application, Security, System, or custom event logs on a computer by using Event Viewer functionality in any of the following applications:

  • Event Viewer. The Event Viewer can be found under Administrative Tools and is the default tool for manipulating the event logs. The Event Viewer gives you the ability to open, view, clear, and save an event log.
  • BizTalk Server Administration console. The Administration console contains an Event Viewer node with the same functionality as the Event Viewer application. It appears after the BizTalk Server Administration node under Console Root.
  • Computer Management. The Computer Management application provides an Event View node with the same functionality as the Event Viewer application, but packages it with many other useful functions including user and group management, share maintenance, storage management, and service control.

You will likely depend on the Administration console to act as your gateway into the Event Viewer, but depending upon where you have deployed the BizTalk Server client tools, you may also use the Event Viewer or the Computer Management application.

Writing to the Event Log

You can also use the Event Log to monitor the progress of the business processes within your BizTalk applications. Information can be written to the Application log or to a custom event log depending upon your needs. Writing to the Event Log can be useful in the following scenarios:

  • You want to track the progress of a complicated business process for troubleshooting purposes.
  • You want to audit key events like provisioning a new user account or denial of certain types of claims.
  • You want to track and respond to certain errors using monitoring software.

If you need to further refine events in your application, you can create a custom event log and write only certain events to it from your application code. This gives you the flexibility to maintain the log on a different schedule than the Application or other built-in event logs.

You must have administrative privileges to create a new event log. For this reason, you should create custom logs as part of your installation process and remove them in the uninstallation process.

Consider using the Microsoft Logging Application Block if you need a flexible, tested mechanism to log events from your application. You can use the Logging Application Block to write information to a variety of locations including the event log, an e-mail message, a database, a message queue, a text file, a WMI event, or custom locations using application block extension points. For more information about the Logging Application Block, see Logging and Instrumentation Application Block.

You can also manipulate the Event Log by using the EventLog class. For more information about the EventLog class, see EventLog Class.

Write confidential or protected message data to the event log only if you have taken steps to ensure that only authorized users have access. Failure to do so may expose confidential data to unauthorized users. For more information about security and event logs, see Security Ramifications of Event Logs.

Event logging consumes resources such as disk space and processor time. The amount of disk space that an event log requires and the overhead for an application that logs events depend on how much information you choose to log.

Use the Network Monitor utility to capture network traffic between BizTalk Server and remote clients or servers. Captured network traffic can be analyzed to diagnose network-related problems.

Network Monitor is available on Windows 2000 Server and Windows Server 2003 through the Add/Remove Windows Components option that is available in Add/Remove Programs.

Network Monitor functionality is provided for Windows XP through the use of the NETCAP.exe utility that is installed with the Windows Support Tools available on the Windows XP CD. For more information about the NETCAP.exe utility, see KB Article 310875.

For information about how to capture network traffic with Network Monitor, see KB Article 148942.

SQL Server Profiler can be used to capture Transact-SQL statements that are sent to SQL Server and the SQL Server result sets from these statements. Because BizTalk Server is tightly integrated with SQL Server, the analysis of a SQL Server Profile trace can be a useful tool for analyzing problems that may occur in BizTalk Server when reading from and writing to SQL Server databases. For information about how to use SQL Server Profiler see the SQL Server documentation.

SQL Server 2000 Query Analyzer and SQL Server 2005 Query Editor can be used to execute SQL statements directly against SQL Server databases. This functionality may be useful for querying the BizTalk Server databases or for updating the BizTalk Server databases in certain scenarios. For more information about Query Analyzer see the SQL Server 2000 documentation. For more information about Query Editor see the SQL Server 2005 documentation.

PSSDiag for BizTalk Server is a troubleshooting tool for Microsoft support engineers to capture diagnostic data. It captures data from many different areas including:

  • BizTalk trace
  • BizTalk HAT information
  • Event logs
  • Performance logs
  • BizTalk registry
  • BizTalk executable attributes
  • BizTalk hotfix information
  • MSInfo details

The program is available on the BizTalk Server installation CD.

To use PSSDiag
  1. Install Tracelog from the Platform SDK available here. Perform a custom installation, clear all features, and then expand the Microsoft Windows Core SDK feature and choose Tools.

  2. Locate the Bin folder under the Microsoft Platform SDK folder, and then copy the Tracelog.exe file to the Support folder under the Microsoft BizTalk Server 2006 installation folder.

  3. Open a command prompt, go to the Support folder, and run PSSDiag.exe /G.

  4. Specify the trace level when prompted: -high or -all, then press ENTER

  5. Reproduce the issue.

  6. Use CTRL+C at the command prompt to stop PSSDiag.

Log files are created in the Output subfolder.

The BTSAssemblyChecker.exe tool is designed to check the versions of assemblies that have been deployed in a BizTalk Server environment and to make sure the correct versions are registered in the global assembly cache (GAC) when you have multiple BizTalk servers in a Management database. When you are using multiple servers in a BizTalk Server environment, it is important to make sure the same version of the assembly has been deployed to all the servers. You can also use this tool to selectively deploy the assemblies to the remote servers using the GAC tool.

BTSAssemblyChecker.exe can be found in the Support directory on the BizTalk Server installation CD.

Most BizTalk Server runtime operations require Microsoft Distributed Transaction Coordinator (MSDTC) support to ensure that the operations are transactionally consistent. If MSDTC transaction support is not available, then the associated BizTalk Server runtime operations cannot proceed. Use the DTCTester tool to verify distributed transaction support across firewalls or against networks. The DTCTester utility uses ODBC to verify transaction support against a SQL Server database and therefore requires that SQL Server is installed on one of the computers being tested. For more information about DTCTester, see KB Article 293799.

Use the DTCPing tool to verify distributed transaction support across firewalls or against networks. The DTCPing tool must be installed on both the client and server computers and is a good alternative to the DTCTester utility when SQL Server is not installed on either computer. For more information about using DTCPing to verify distributed transaction support, see KB Article 306843.

RegMon, FileMon, and DebugView are three free utilities available from Microsoft.

  • RegMon displays registry access activity in real time, listing each call that an application makes to the registry and logging the outcome. This tool allows you to identify when an application cannot access a registry key.
  • FileMon displays file system activity in real time, listing each system call that an application makes and registering the outcome.
  • DebugView lets you monitor debug output on your local system, or on any computer on the network that you can reach through TCP/IP.

By using RegMon and FileMon, you can test an application and identify any failed calls that the application makes to the registry or file system. If there is a failure, it can be mitigated by changing file system or registry key permissions or by modifying the application.

The DebugView utility captures the output of the .NET methods Debug.Write and Trace.Write and their variants. It can be a powerful tool both early in development when trace/debug statements can be used liberally and late in development and into production to pinpoint errors to a specific code block.

For more information about these utilities, see the Sysinternals Web site at

The Debug Diagnostics Tool of the Internet Information Services (IIS) Diagnostics Toolkit can generate a memory dump of a failing process and perform a basic analysis of the generated dump file.

The BizTalk Server 2006 Best Practices Analyzer performs configuration-level verification by reading and reporting only. The Best Practices Analyzer gathers data from different information sources, such as Windows Management Instrumentation (WMI) classes, SQL Server databases, and registry entries. The Best Practices Analyzer uses the data to evaluate the deployment configuration. The Best Practices Analyzer does not modify any system settings, and is not a self-tuning tool.

To download the Best Practices Analyzer, see BizTalk Server 2006 Best Practices Analyzer.