Developing Integration Solutions with BizTalk Server 2004

Angus Foreman; Andy Nash

Microsoft Corporation

March 2005

Applies to: BizTalk Server 2004

Summary: "Developing Integration Solutions with BizTalk Server 2004" provides developers with techniques for designing, developing, and deploying solutions within Microsoft® BizTalk® Server 2004. (88 printed pages)

Download sample code

This guide contains examples of team approaches for developing with BizTalk Server 2004, and illustrates techniques that can increase development efficiency. It also provides hints and tips that can decrease the time and effort spent debugging and testing. It emphasizes the team development process and team development setup.

This guide focuses on projects that use the core messaging and orchestration functionality of BizTalk Server. It does not cover in detail additional areas such as Business Activity Services (BAS), Business Activity Monitoring (BAM), Human Workflow Services (HWS), XREF, or the Business Rules Engine (BRE).

The following users should read this guide:

  • Solution designers will gain understanding of the constituent parts of a BizTalk solution and the possible project structures used to develop BizTalk Server 2004 solutions.
  • Developers will gain an understanding of the process of developing a BizTalk Server 2004 solution in a team and learn techniques that can improve developer efficiency.
  • Project managers will gain an overview of the main project phases and understand typical tasks when planning a BizTalk Server 2004 team development.

This guide assumes that the reader is familiar with BizTalk Server 2004 and understands the core concepts of the product.

The document is based upon a subset of the project phases that are described in the Microsoft Solutions Framework, namely:

  • Planning
  • Building and Developing
  • Stabilizing
  • Deploying

The Microsoft Solutions Framework (MSF) contains proven practices for planning, building, and deploying small to large information technology (IT) solutions. MSF is made up of principles, models, and disciplines for managing the people, process, and technology elements that are encountered on most projects. For more information about MSF, see the Microsoft Solutions Framework Web site at http://go.microsoft.com/FWLink/?LinkID=42401.

This section describes a lightweight technique for gathering integration requirements and translating them into a list of BizTalk Server 2004 artifacts to be delivered. It discusses a typical BizTalk Server 2004 integration scenario. Readers can compare this typical scenario with their own requirements and create plans and documents accordingly.

The planning phase also includes setting up the development environment and the development processes to provide a stable and efficient development environment. Making changes to the development environment or processes during the development phase can be very disruptive, so time spent planning them is likely to save time later in the project.

Planning for the integration scenario includes the following:

  • Gathering information
  • Defining naming conventions
  • Planning team development
  • Setting up and working with source control

Gathering Information

Like most software development problems, the successful delivery of an integration solution depends upon the collection and collation of large amounts of information prior to the design process. The following sections suggest a lightweight approach to gathering the information that is typically useful to the design and development of an integration solution. This information includes the business processes and the messages that the solution requires.

Business Processes

An important function of the planning process is to draw up the master list of business processes that you will deliver with the integration solution. This task ensures that a single, comprehensive list of all the business processes exists, along with a list of all the business rules that apply to those processes. You can use this list to plan and design the development of the BizTalk Server 2004 artifacts that are required to carry out those business processes.

One possible approach to developing the list of business processes is to devise logical categories of related business processes and then list all the processes within each category. For example, categories like "User Management," "Supplier Transactions," "Customer Transactions," "Catalogue Management," or "Credit Control" are often meaningful to the business and typically consist of several related business processes (with related system and messaging requirements).

In the example scenario, the business process list is documented in the following tabular format.

Table 1 Example processes and categories

Business process category Process name

Account Management

AccountCreateNewAcc

AccountUpdate

AccountValidate

Order Fulfillment

OrderProcessNewOrder

OrderConfirmBillingStatus

Billing

BillGenerateMonthlyBill

BillGeneratePendingBill

BillUpdateBillHeader

The team now has a master list of business processes that need to be developed. The next step is to capture the process details that are significant to the integration solution. Detailed information that is typically captured about each process falls into two categories—logical process information and implementation information. You may also want to diagram the high-level processes so that developers will have a starting point for creating BizTalk orchestrations.

Logical Process Information

This information describes the process in the abstract, without being concerned about the actual implementation technologies. It is typically gathered by interviewing the owners of the business processes rather than IT or system specialists. To gather logical process information, you might perform actions or answer questions like the following:

  • Produce a "flow chart"-style description of process logic. See "Diagramming Business Processes" below.
  • Define core business entities (not systems) that are involved in the process, for example, customers, orders, parts, invoices.
  • Identify the events or actions that initiate a process (both human and system-based).
  • Determine where exceptions occur in the current business processes and how those exceptions are handled (that is, is a process restarted, is human intervention required).
  • Determine where business data validation takes places in a process.
  • Understand the business rules that form part of the process.
  • Determine whether there are long-running processes (for example, days, or weeks).
  • Is any human intervention required to complete a process?
  • What is the nature of the communication between processes and other systems? For example, is the process real-time or batch-based?
  • Does information need to be translated or transformed?
  • Are manual processes being automated?
  • Are new processes being created to achieve the solution?
  • Are any of these processes currently residing in other systems? Can these be documented?
  • What business metrics, milestones, or key business data in the process must be reported to management?
  • How are humans notified or how do they interact in the process?
Implementation Information

Information related to the implementation is often dictated by the design and constraints of the existing systems being integrated. It is important to keep in mind the constraints placed around processes by the environment and other systems. These constraints often significantly influence the final design.

To gather this information you might perform actions like the following:

  • List the transport-specific requirements for each process.
  • List the security models and protocols required to interact with other systems.
  • List the messages to be consumed and produced by a given process.
  • Understand the message format constraints that exist (that is, which messages exist, which are standards, and which need to be created).
  • Detail the requirements for handling exceptions and errors.
  • Understand any transactional data requirements of a process, including any compensation logic required for long-running transactions.
  • List any message correlation requirements.
  • List any requirements for cross-referencing the indexing of data between the systems being integrated. (For example, is it necessary to maintain a link between the two separate customers? Do IDs used by different systems describe the same customer?)
  • List the auditing or management information system requirements.
  • List the applications or interfaces used to allow human intervention into the processes.
  • Understand the operational requirements of the processes, that is, what operational data must they expose (for example, logging) and what operational systems must they comply with?
Diagramming Business Processes

The Orchestration Designer for Business Analysts (ODBA) is a Microsoft Visio®-based tool that provides an environment for designing business processes as orchestrations. This tool is intended for business analysts to create basic orchestration flows in the familiar Visio environment. Using this tool, business analysts can create high-level orchestrations that can later be exported to the Orchestration Designer that developers use for implementing orchestrations.

The ODBA has the following advantages for discussing, white-boarding, and designing orchestrations:

  • It uses Visio (rather than Microsoft Visual Studio® .NET) to run, and is therefore better suited to non-developer environments.
  • It can be used by non-developers who are familiar with Visio.
  • The resulting diagrams can be easily incorporated into requirement and design documents.
  • The output from the ODBA can be imported into Visual Studio .NET and used as the basis for orchestration design.

To download the Orchestration Designer for Business Analysts, go to the downloads section of the BizTalk Server Web site at http://go.microsoft.com/FWLink/?LinkID=42402.

The use of Unified Modeling Language (UML) sequence diagrams or the "swim lane" style of diagrams can be useful in gaining a consensus on the actual flow of a business process. Microsoft Office Visio provides specific UML sequence shapes that can be used to document and distribute business process definitions. For information on creating UML sequence diagrams in Visio see the Office Online Web site at http://go.microsoft.com/FWLink/?LinkID=44996.

Defining Messages

The process of examining all business processes produces a list of the messages that are required to deliver the solution. When defining messages consider the following:

  • Determine the complete list of entities that are meaningful to the organization to allow the processes to be designed. Typically you started this list when looking at the processes and you will refine it here to produce the "business-centric" list of all the entities that are required.
  • Determine which message schema will define the "standard" messages. These are messages that are not constrained by any specific integration requirement but instead define a business entity, concept, or operation.
    As an example, consider a "customer" message. Typically, different back-office systems implement interfaces that handle many different subsets of customer data, but there is only one standard that represents the organization's view of "customer." Note that this standard is based upon how the organization views the entity (as opposed to a system). This information could then used to define the Customer Message Standard for the organization and will be driven by the organization's long-term goals. For example, you can define the customer message so that it can contain data that is not currently required, but will be needed to deliver new services in the future.
  • When considering standards for messages, consider the cost of creating a standard and the cost of mapping to and from the standard for each of the upstream and downstream systems. In some cases, the standard provides benefits (especially when multiple systems all use a variation on the standard). In other cases mapping from one system to another system (without mapping to the intervening standard) can be a more practical approach.
  • Determine the ownership of standards. It is important for an organization to realize that messaging standards are a representation of business entities and concepts and should be owned by the business and IT jointly. This includes the owning of namespaces.
  • Are there existing messaging standards that you can use? Industry-specific standards may exist for exchanging data between organizations (for example, health care and finance). It is worth determining whether such a standard exists, and if it does, whether using the standard rather than defining a new one can provide benefits (both current and future).
  • Some vendors may publish standard messages for interacting with their systems. Using these schemas may save considerable development effort and remove specific integration challenges from the project team to a vendor's supported solution.
  • When defining messages, your result should be a comprehensive list of the messages you require to implement the solution. By stabilizing and gaining agreement to this list early in the project, you can avoid the introduction of new messages late in your development phase.

Defining Naming Conventions

When describing the complexities of an integration solution, it is important that you use a strong set of terms to ensure clarity throughout the planning, designing, and development phases. You begin this process in the planning phase by identifying the naming conventions you use to describe systems and their operations. The naming conventions in the following table can be applied to the integration solution.

This document does not present a comprehensive list, but instead provides terms found useful on a variety of projects. For a comprehensive approach to integration terms, see the MSDN® paper "Enterprise Integration Patterns with BizTalk Server 2004" at http://go.microsoft.com/FWLink/?LinkID=42403.

Table 2 Integration terms

Name Description

Integration layer

The technology that provides the actual integration functionality. In this document, the integration layer is provided by BizTalk Server 2004.

Upstream system

A system that participates earlier in the business process than the point currently being discussed or documented. Typically upstream systems transmit messages to the integration layer.

Downstream system

A system that participates later in the business process than the point currently being discussed or documented. Typically downstream systems accept messages from the integration layer.

Source system

The system from which a specific message originates.

Destination system

The system to which a specific message is targeted.

Process initiator

A system or event that starts (initiates) a process that the integration layer will be involved in.

Activated process or activated system

A process or system that is activated by an initiating process.

Master data

Data that is the master copy for a given process.

Adapter

Messages from a system are sent by using some protocols handled by an adapter. An adapter abstracts the system from the protocols used to communicate.

Duplicated data

Data that is the duplicated from the master for a given process. Sometimes known as slave data.

When describing data or messages between systems and the integration layer, it can also be useful to use a standard notation to avoid confusion over the direction of message flow. This is particularly useful when multiple request-response operations exist between systems.

One possible approach is to describe message instances between the systems using the notation of "incoming" and "outgoing" where "incoming" and "outgoing" are understood from the point of view of the integration layer. The following messages are examples:

Messages sent to the integration layer:

  • CRM_Add_Customer_Incoming_Request
  • Order_ProcessOrder_Incoming_Response

Messages sent from the integration layer:

  • Order_ProcessOrder_Outgoing_Request
  • CRM_Add_Customer_Outgoing_Response
  • The following figure shows the messages targeting the integration layer.

Figure 1 Message direction and naming

n/a

While it can be helpful to name message instances in an orchestration, naming the message schema with a name describing direction may cause confusion in other areas of the solution, especially if the schemas are reused or exchanged with other parts of the system.

Many line-of-business messaging applications, especially financial systems, use a numeric system for identifying the messages (that is, the message type, or schema, and the destination system/direction), for example, 9820000-1104 in ISO 8583, MT 512 in SWIFT. In such cases you can either maintain the numeric notation or use a combination of numbers and text to identify messages (such as MT_512_Securities_Trade_Confirmation).

It is not uncommon for a complex process to use 10 to 20 related incoming and outgoing request and response messages. A naming convention applied to orchestration messages and variables can aid clarity. A standardized approach can reduce possible errors and confusion and make documentation more effective.

In some cases, naming conventions can produce very long names that are not easy to use in Visual Studio, given that the dialog boxes and panes are narrow. In these cases consider using a coding scheme such as CRM_Add_Customer_IReq and CRM_Add_Customer_Oresp, where IReq is an incoming request and OResp is an outgoing response.

Describing Systems

When describing an integration solution, there is typically a large amount of detail to be captured about each of the systems and processes. It is useful to start by listing some common properties of each of the systems involved. It is typical to collect this information for each system participating in the integration solution.

The following table shows some common system attributes.

Table 3 Common attributes of systems

Attribute Description

System purpose

The general business and technical purposes of the system being documented. Often there are system specialists involved in an integration solution who have no previous knowledge of the other systems involved. An overview can be valuable to ensure effective communication across the team.

Interfaces and access methods

A description of how the integration layer accesses the system (and vice versa). This is usually described in terms of the protocols and standards used by the system for both incoming and outgoing operations.

Message and data formats

Details on the likely format of incoming and outgoing data and messages.

Data master

When understanding data and process it is important to understand which system holds the master copy of each type of data. For example, a CRM system may be the master for all customer address data, but may also hold replicated accounts data.

Volume and frequency of data operations

The volume and frequency of data operations can have an impact on the design of an integration solution. During planning, it is typical to capture as much information about this volumetric data as possible. Ensure that these figures include peak volume requirements because these may be a significant requirement.

Security models used

When designing an integration solution, the security requirements are often one of the most complex areas to model and implement. Typically this information consists of:

  • A description of the authentication and authorization models imposed upon the integration layer when calling into a system
  • A description of the authentication and authorization models used when calling out of a system into the integration layer
  • Any requirements the system places on secure communications (for example, encryption required to communicate with the system)
Example Integration Information Gathering Scenario

In our example scenario the planning process starts by documenting the systems as shown in the following table, with more detail being included as the planning and design processes uncover more information. The following table lists examples of the type of data that is typically recorded. The systems described here represent a customer relationship management system and an account management system.

Table 4 Example system information

System Customer relationship management system (CRM) Account management (Accounts)

System purpose

Provides pre-sales, sales, and customer management functions to customer services representatives. The application provides the logic, storage, and front end for all interaction between customers and the organization (for example, the screen used in call centers when a customer calls up). Used to manage prospects and to manage the status of a customer.

Maintains the databases and logic used to manage customer accounts and the logic to run account billing and management processes.

Interfaces and access methods

Incoming and outgoing: Access to and from this system is via custom XML documents posted using HTTP over Web interfaces (not Web services). The XML documents are defined by the CRM system. That is, other systems wishing to communicate with the CRM system must conform to the specifications laid down by the CRM system.

Typically used in an interactive manner, where a change needs to be made to a customer or product within a matter of seconds or minutes. (Synchronous).

Incoming: Unidirectional HTTP access to the system, with the system accepting XML messages on an HTTP listener, with no response message (other than a transport success message).

Outgoing: Batch-based file outputs typically running nightly, weekly, or quarterly processes. Files written to a UNIX file share location.

Message and data formats

Incoming message formats: CRM-specific XML schema. Definition available as XSD file. Actual schema used is dependent upon message type (of which six exist).

Outgoing message formats: CRM-specific XML schema. Definition available as XSD file. Actual schema used is dependent upon message type (of which six exist).

Incoming message formats: Custom XML message as defined by Accounts system. Sample XML messages available.

Outgoing message formats: Positional ASCII flat file with headers and footers. Definition available as Word document.

Data master

CRM is the master source for pre-sales prospect information.

While the system provides access to customer information, it is not the master for customer information (which is held in Invoicing).

This system is the master for customer and account information.

Volumes and frequency of data operations

Updates from the CRM system are regular during the day, covering the entire 24-hour period (due to a global roll out). Volumes of operations are 1-2000 per day. The resulting 1-2000 messages mostly hold customer data. Due to the internal definition of a customer used by the CRM system the messages to and from the CRM system can consist of very large XML documents, with sizes of 500K not uncommon. Message size and frequency is significant parameter in sizing the integration servers and may be significant during the design phase.

Incoming: Incoming requests to the Accounts system are infrequent and small in size.

Outgoing: Due to its batch nature, under certain circumstances the volume of data passed from the Accounts system can be very large. Typically these large volumes occur during processes like quarterly or year-end billing runs. These large volumes are significant because it is often the case that there exists a relatively short duration in which all this information must be processed. (For example, there might be 1000 files of 3 MB each.)

Service windows

Available for a 24 x 7 global application.

Available GMT 8 AM – 8 PM (with peak usage at month ends).

Security models used

Incoming: The CRM system expects a user identity and password to be passed as a part of all incoming messages for validation against an internal user store.

Outgoing: By default the CRM system's outgoing HTTP requests do not pass credentials to the target system.

Incoming: HTTP requests accepted from a set of known IP addresses only.

Outgoing: Configurable credentials are used to write outgoing data files to a specific location.

Categorizing Interfaces

Having previously defined the systems and business processes in scope, the next step is to document the interfaces that are required to enable the integration layer to connect to the upstream and downstream systems. One way to capture this information is to list the processes and then detail each of the interfaces required by each process. If you are designing a system to replace an existing integration solution it can be helpful to obtain actual copies of all the types of messages that are currently going through the various systems.

The following list describes typical information that you should gather while you are categorizing interfaces:

  • Interface name
  • Business name
  • Interface description
  • Source system name
  • Destination system name
  • Interface address (URI/URL)
  • Interface direction—examples include:
    • Unidirectional/Incoming (to the integration layer)
    • Unidirectional/Outgoing (from the integration layer)
    • Bidirectional/Incoming
    • Bidirectional/Outgoing
  • Protocol
  • Message/data format used
  • Message schema name
  • Expected number of messages per minute, average and peak
  • Expected average message size
  • Message schema owner

Later in the planning process, this list of interfaces can be further extended to help produce a detailed list of functional requirements that can be used as the basis of a BizTalk Server 2004 development.

A helpful way to document those functional requirements is in the form of test cases with specific input files and expected output files. This typically means collecting a lot of data early in the project, but in all likelihood this data will be required when development or testing starts. By gathering it early in the development process, you can spot any unanticipated challenges early on.

Documenting Interfaces as Contracts

When developing an integration solution, the definition of interfaces is a significant part of the design process. The interface is typically considered as a "contract" between each of the parties involved in the development of the integration solution. The contract verifiably ensures that parties are developing compatible systems.

When documenting interfaces it is useful to be able to produce detailed descriptions of the interfaces as part of the contract. The best way to document the contract is with a domain-specific language like Web Services Description Language (WSDL), or with some other consumable form of metadata. All parties can use these descriptions to validate that their development meets the interfaces expected by upstream and downstream systems. When working with XML-based solutions like Web services and BizTalk Server 2004, standards exist to produce these detailed contract descriptions. The following table shows the contract formats for various interface types.

Table 5 Interface types and contract formats

Interface message type Contract format

XML message

XSD schema files.

ms942199.note(en-US,BTS.10).gifNote
If no XSD schema file exists, but sample XML messages exist, then BizTalk Server 2004 can generate an XSD based upon a sample XML file. See "Add Generated Items" in BizTalk Server 2004 Help for more details on this functionality.

XML Web service

WSDL definition files.

Flat files

BizTalk schema and the flat file validation tools.

ms942199.note(en-US,BTS.10).gifNote
The flat file command-line tools can be used to aid the validation of flat file instances against a BizTalk Server schema and can be found in the SDK under SDK\Utilities\Pipeline tools.

Summary

At the end of the information-gathering process, the high-level requirements of the integration solution are captured in a series of related documents that list unique business processes, systems, and interfaces. As the design process progresses, the level of detail captured increases. Prior to development, it is typical to produce a list of BizTalk Server 2004 artifacts to be developed during the build phase.

Additionally the planning phase may well identify the number and type of test harnesses or stub systems that will be required to meaningfully develop and test the business processes. The planning phase may also identify the data requirements needed to allow development to proceed. This may include migration requirements and cross-reference data.

You use this list of requirements to aid in designating teams, planning schedules, and identifying required development resources.

Planning Team Development

The approaches described in this section relate to the general approaches needed when developing within a team. Many of the concepts in this document are covered for general .NET development in the MSDN white paper Team Development with Visual Studio .NET and Visual SourceSafe.

Dividing Tasks and Allocating Ownership

When developing a solution within a team it can be very helpful to define owners and development teams for distinct parts of the solution early in the development process. Typically, named individuals (or groups of individuals) are given ownership of logical parts of the solution. Useful divisions of work can include allocating ownership of:

  • Individual business processes
  • Process-specific messages, schemas, transports, and protocols
  • Shared message standards
  • Map development
  • Error messages and error-handling standards
  • Upstream and downstream system interfaces
  • Helper classes and utilities
  • Test data and test tools
  • Deployment and build processes

Typically, the owners are responsible for understanding their parts of the solution and act as the "gateway" to any requested modifications. In this way when a modification to a design is proposed, the owner can assess the effect of the proposed changes and then allow or disallow that change. This is particularly important when working with shared parts of the solution, like shared message standards, where a change can have a potentially significant impact upon the development team and upstream or downstream systems.

Solution Structures and Team Development Models

The following sections discuss solution structures and development models for a BizTalk Server 2004 project with multiple developers.

The term "solution structure" refers here to the logical and physical structure used to store all the related BizTalk Server 2004 files and folders required to produce, develop, test, and deploy a production solution.

The term "team development model" refers to the relationship between Visual Studio .NET projects and solutions, Microsoft Visual SourceSafe® source configuration, and the setup and processes used for a team of developers using common resources across multiple workstations.

It is important to enforce a solution structure. A BizTalk Server 2004 solution that is built to meet business requirements typically consists of many separate related artifacts. To avoid rework and build failures, these artifacts need to be managed throughout the project from development, test, deployment, and production. A typical BizTalk Server 2004 solution will contain the artifacts in the following table.

ms942199.note(en-US,BTS.10).gifNote
This list does not include artifacts related to Business Activity Monitoring (BAM), Human Workflow Services (HWS), and the Business Rules Engine (BRE).

Table 6 Typical Visual Studio .NET and BizTalk solution entities

Artifact Description

Visual Studio .NET solution files

The files produced by Visual Studio .NET when creating and editing a BizTalk Server 2004 solution.

Strong name keys

Strong name key files used to assign strong (unique) names to the assemblies produced during the build process.

Binding files

The files that BizTalk Server 2004 uses to bind a BizTalk Server 2004 assembly to physical resources (HTTP addresses or file locations).

ASP.NET Web service files

ASP.NET files, .NET DLLs, and configuration files produced by the BizTalk Web Services Publishing Wizard that provide a Web service interface to BizTalk Server 2004.

Output DLLs

The assembly files produced by the build process.

HTTP receive functions

Configuration information that describes Microsoft Internet Information Services (IIS) virtual directories that host the BizTalk Server HTTP receive DLL.

Referenced DLLs

Files containing functionality used by the BizTalk Server 2004 solutions using file references.

Cross-reference data

Data used to prime the BizTalk Server 2004 cross-reference databases.

Test data

Data files used to initiate or participate in the testing of processes under development.

Test tools

Tools used to initiate or participate in the testing of processes under development. These might include "stub" Web services that act like a Web service not available during testing, or tools that act like an external application submitting a message into BizTalk Server 2004.

The solution structure is important to a team of BizTalk Server 2004 developers for the following reasons:

  • Any integration solution by its nature consists of many independently developed assemblies that need to fit together, like pieces of a jigsaw puzzle, to deliver the complete solution. If these pieces are not closely managed throughout the development, there is a strong chance the pieces will not fit together when the complete solution is tested. A predefined solution structure assists in enabling the management of these parts.
  • In a solution of medium to high complexity, BizTalk Server 2004 projects are themselves often linked and dependent upon each other. A solution structure helps ensure that the relationships between BizTalk Server 2004 projects is enforced and maintained throughout the development process.
  • Deploying and testing a BizTalk Server 2004 solution requires more than just the output of the compilation process. A typical process may require binding files, test data, test harnesses, and stub systems as well as the BizTalk Server 2004 assemblies to perform a functional test. A well-defined solution structure allows testing and deployment to be automated.

A typical BizTalk Server 2004 solution structure is composed of the following logical elements:

  • A Visual Studio .NET solution structure, which describes the relationship between BizTalk Server 2004 solutions, project files, and dependencies.
  • A file system folder structure in which to store BizTalk Server 2004 project files and the other associated entities required to develop and test a solution.
  • A development model that describes how a team of developers share the server resources required to develop a solution.
  • An approach to using Visual SourceSafe that enables the BizTalk Server 2004 projects and artifacts to be successfully source controlled and shared among a team of developers.

Dividing BizTalk Server Artifacts between Projects

There are many ways to create a BizTalk Server application composed of maps, schemas, orchestrations, pipelines, and so on. The most common is to place all of the BizTalk artifacts in one project, compiled and then deployed as a single unit. This method has the benefit of simplicity but is less effective when you take into account required modifications and project upgrades.

Using this model, when modifications or upgrades are needed, you must first undeploy the complete application. This means stopping the orchestrations and potentially affecting related entities. For example, if you use custom pipelines, the undeploy operation causes the pipeline bindings to default back to the pass-through pipeline. After you deploy a change, you must reconfigure any location that uses custom pipelines.

As an alternative to using the single-project approach, consider how the application is structured and look for ways to split the artifacts into logical groups. A typical way to manage this is to place orchestrations, pipelines, schemas, and maps in their own projects. If a pipeline changes, just a single assembly can be undeployed, and there is no need to undeploy maps or schemas. In the same way, if a map changes there is no need to undeploy the schemas. Additionally, consider dividing sub orchestrations (that is, orchestrations that are called by other orchestrations) into their own projects.

Remember that a project can only be deployed as a single unit. If the task of redeploying a single project begins to involve the rebinding of unrelated resources, then you should consider splitting the project into smaller units that allow the deployment and binding of related items only.

ms942199.note(en-US,BTS.10).gifNote
The preceding approach is a "horizontal" division of work. You may want to consider a vertical division as well (that is, by project or business process functionality). For example, if a pipeline or schema is updated it should only affect the business project that it is related to. This is relevant if you have the requirement to avoid shutting down a CRM integrator when the ordering system updates a schema.

The following table shows the structure of a typical project.

Table 7 Typical project structure

Project Description

Standard schemas project

("Shared Schema")

This project contains all the schemas that are shared across the whole solution. This project is referenced by other BizTalk Server 2004 projects. It is separate because it needs to be shared and should be stabilized early in the project.

Functionally grouped orchestrations ("Billing Project")

These projects contain logically related orchestrations, such as "Billing project" and "Invoicing project." These are usually separated out because these related orchestrations are usually under control of a single individual and they often contain schema local to their own processes.

Shared orchestrations projects

("Email Handler")

These are orchestrations that are shared across the whole solution. These are kept separate to allow other projects to reference them. Typical examples include "error handling project" or "email send project." They contain the orchestrations and schema necessary to complete a discrete set of tasks. Where possible, you should stabilize these functions early on in the project because changes can affect all projects using them.

Team Development Models

Team development with BizTalk Server 2004 has close parallels with team development of Web applications. Both models typically have dependent elements that can be shared between the team members or a user in isolation.

The development model for BizTalk Server 2004 proposed in this paper is based upon the "isolated model" of development. The models available for BizTalk Server development are "isolated," "semi-isolated," or "non-isolated." In the isolated model, no elements of the developer environment are shared (with the exception of access to a common Visual SourceSafe tree). When using semi-isolated and non-isolated models, increasing areas of the developer environment are shared. The nature of the BizTalk Server 2004 runtime means that isolated development typically provides the most effective approach. BizTalk Server 2004 developments are typically composed of several distinct environments, as shown in the following table.

Table 8 BizTalk Server environments

Environment Description

Developer tools

Tools used to produce BizTalk Server 2004 solutions. Composed of Visual Studio .NET and BizTalk Server 2004 design-time tools and test tools.

Developer runtime

Environment used to test a developer's output. Composed of the BizTalk Server 2004 runtime, deployment tools, and test tools.

Integration test environment/shared test runtime

Environment used to test multiple developer deliverables in a single environment (often logically mirroring the production environment). Composed of the BizTalk Server 2004 runtime and deployment tools.

Pre-production environment

Environment that is as close to the physical production environment as possible. This is used to test the deployment processes and runtime of the completed integration solution. This environment can also be used for scalability and performance testing.

Composed of the BizTalk Server 2004 runtime and deployment tools.

Production environment

Environment used to deploy the completed integration solution. Composed of the BizTalk Server 2004 runtime and deployment tools.

The team development model described below details an approach to configuring the developer tools, developer runtime, and shared runtime to provide an isolated developer environment.

What Is the Isolated Development Model?

Using an isolated model a developer edits, debugs, and runs functional tests completely isolated from other developers. Each developer has a self-contained development workstation with a local BizTalk Server Group. Access to the master source files is controlled via a Visual SourceSafe database located on a network file share. The following figure illustrates an isolated development model.

Figure 2 Isolated BizTalk Server development

ms942199.Local_-716283123_figure02(en-US,BTS.10).gif

The isolated model of BizTalk Server development provides the following benefits:

  • No chance that BizTalk Server shared configuration information will interfere with another developer's work (for example, XML target namespace clashes occurring from multiple installed versions of shared schema)
  • No opportunity for any one individual's BizTalk Server deployment and test processes interrupting other users (for example, the starting and stopping of BizTalk Server host instances, receive locations, or dependent services like IIS, SQL, and SSO)
  • Ability to clear resources like event logs and tracking databases without disrupting other users
  • Ability for developers to use different versions of shared resources, for example, helper classes and error-handling processes
  • Ability for developers to attach and debug the BTSsvc.exe process without halting other developers' processes

A shared or non-isolated model can be used for developing with BizTalk Server 2004 if it is absolutely required, but developer efficiency may be reduced by the lack of the benefits listed above.

Using Microsoft Virtual PC to Host BizTalk Server 2004 Development Environments

In our experience of developing BizTalk Server 2004 solutions Virtual PC can be used to efficiently create an isolated development environment. Within Microsoft Consulting Services, this is the standard approach used on nearly all BizTalk Server 2004 developments. Using Virtual PC typically saves the development teams a considerable number of hours of effort (per developer) to create the development environment, and is an extremely efficient way to enforce a standard development environment. It has additional advantages including the ability to efficiently "package up" a development environment. This technique is often used for scenarios like sharing best practices, handing over issues between teams, and testing possible server configurations.

For more details on implementing this approach, see "Using Virtual PC or Virtual Server to Host the BizTalk Server 2004 Developer Environment" later in this document.

For product information about Microsoft Virtual PC 2004, see the Windows Web site at http://go.microsoft.com/fwlink/?LinkId=44997.

Partitioning Visual Studio .NET Projects and Solutions

Visual Studio solution files (with the .sln file extension) are used to group related projects together and are primarily used to control the build process. You can use solutions to control build dependency issues and control the precise order in which contained projects are built. Partitioning describes the processes of creating Visual Studio .NET solution files that divide up the multiple projects that compose a complete BizTalk solution. The approach used to partition BizTalk Server 2004 solutions and projects has a significant impact on the development and build processes in a team environment. There are three main approaches to consider when partitioning solutions and projects, as shown in the following table.

Table 9 Solution partitioning approaches

Partitioning approach Description Applicability

Single solution

The team creates a single Visual Studio .NET solution (the master solution) and uses it as a container for all of the BizTalk Server 2004 projects required by the BizTalk Server 2004 solution.

Viable for simpler BizTalk Server 2004 projects with only two or three developers.

Partitioned single solution

Used in more complex developments where related sets of projects are grouped together within separate sub-solution files. A master solution still exists but is used for building the entire solution on the build server.

Should be considered for more complex solutions with several developers.

Multi-solution

Using this model there is no master solution file and file references are used between projects in separate solutions (although project references are still used between projects within an individual solution).

Not recommended. Relies on non-recommended file-based references.

The partitioned single solution approach is often applicable, but is the most complex to set up. For these reasons this approach is documented in the appendix. The single solution approach uses the same principles as the partitioned, and the approach documented can easily be modified for the single solution. Unless you have very good reasons to use a multi-solution model, you should avoid this approach.

Attributes of a Partitioned Single Solution

When developing more complex BizTalk solutions it can be advisable to reduce the number of projects and source files required on each development workstation by grouping related sets of projects together within separate sub-solution files. This approach can be particularly beneficial if the complexity of the BizTalk Server 2004 solution means that loading, navigating, and rebuilding projects can take a significant amount of time.

This partitioned single solution allows developers to work on separate, smaller subsystems within the inner-system boundary. The following diagram illustrates the partitioned single solution model. Notice how separate solution files are used to allow you to work on smaller subsystems contained within the inner-system boundary. Also note how this results in projects being contained within more than one solution file. For example, in the following figure, Projects D and H are in a total of three solution files including the master solution.

Figure 3 Partitioned single solution

ms942199.Local_-716283124_figure03(en-US,BTS.10).gif

In the partitioned single solution model:

  • All projects are contained within a master solution file. This is used by the system build process to rebuild the entire system. If you need to work on the top-level project file, you also work with the master solution.
  • Project references are used between individual projects. For example, file references are not used and instead the project containing the dependency is added to the solution and a project reference created.
  • Separate solution files are introduced for selected project files. If you want, you can introduce a solution file for each project within your system. Each solution file contains the main project file, together with any downstream project it depends on and any further projects it depends on, and so on down the dependency chain.
  • Separate solution files allow you to work on smaller subsystems within your overall system but retain the key benefits of project references. Within each sub-solution file, project references are used between constituent projects.

For more information about partitioning Visual Studio .NET solutions see Chapter 3, "Structuring Solutions and Projects," in the MSDN white paper Team Development with Visual Studio .NET and Visual SourceSafe that was referenced at the beginning of this section.

Using Project References

As described in the previous section, single solutions and partitioned single solutions use project references to create references between dependent BizTalk Server 2004 projects. This is significant because project references provide significant advantages over file-based (DLL) references. Specific benefits include:

  • A build process using project references automatically calculates the build order dictated by the dependencies. This significantly simplifies the build process for projects with dependencies.
  • A project-referenced solution can determine if a referenced project has been updated and will manage the rebuilding of the references. This is important in an environment when several users have dependencies upon a shared project (like a helper project) that is changing during the development phase.
Referencing External Dependencies

Use DLL references when referencing an external DLL that is outside of the BizTalk Server 2004 solution.

For example, suppose that an external organization is supplying schemas that are out of the control of the BizTalk Server development project. These DLLs would be referenced by DLL. Other examples are .NET Framework assemblies and third-party assemblies.

Copy Local Attribute

When working with project or file references, do not change the default "Copy Local" attributes for a referenced project or DLL.

For more information about partitioning Visual Studio .NET solutions see Chapter 4, "Managing Dependencies," in the the MSDN white paper Team Development with Visual Studio .NET and Visual SourceSafe that was referenced at the beginning of this section.

Creating the VSS Structure

The file system folder structure used to store the BizTalk Server 2004 integration solution must accommodate not only the BizTalk Server 2004 solutions and projects, but also all the additional artifacts that are required to develop, build, test, and deploy a BizTalk Server 2004 solution. The folder structure must also be compatible with the structure used within the team's Visual SourceSafe (VSS) repository to ensure efficient integration with the common source control tasks.

The folder structure described here is based upon typical BizTalk Server 2004 requirements and is designed to comply with common Visual SourceSafe tasks. It can be modified to meet the requirements of a given development team.

When creating the VSS structure, keep in mind the following points:

  • Use a consistent folder structure for solutions and projects. Development in a team development environment is easier to manage if a common structure for storing Visual Studio .NET solutions and projects is used across the whole team. This is especially true when each BizTalk Server 2004 project will be generating and using files such as binding files that will be required by build, test, and deployment teams.
    The structure of BizTalk Server 2004 project files means that certain project entities (like the strong name key file) are by default referenced by an absolute file path. This means that unless the same file structure is replicated by each developer, each developer will need to modify the project to allow the project to compile and deploy.
    It is often easy to start development without taking the time to standardize on the folder structures. This will inevitably cost more time later on as project links and test harnesses need to be "patched" as the solution develops.
  • Keep VSS and file system structures identical. To simplify the management of multiple developers' environments, set up a folder structure within Visual SourceSafe that matches your local file system structure. This helps ensure that a developer can simply get a latest version from Visual SourceSafe and know that the structure on the disk is compliant to the team's standards.
  • Define and use a common root folder across the team. It is recommended to keep all project code and deliverables under a single folder root, which can be created on all developers' computers. For example, create a root folder of $/SysInteg2004 within Visual SourceSafe and all developers can create C:\SysInteg2004 on their local file systems. This acts as a container for all of your development systems.
    The drive need not be the C drive, but if a drive other than C is used, make sure that all developers have a similarly named drive available. In case you want to use drive letters that are not available on all computers, the SUBST shell command can be used to create drive letters mapped to physical folders, for example:
    SUBST m: c:\SysInteg2004
    
    When the preceding command is executed at the command prompt, it creates a drive letter m: mapped to physical folder c:\SysInteg2004. Avoid using a network share to store the project files (to avoid the security warning that arises from developing a .NET application on a non-local drive).
  • Create a master solution that will hold all projects. As described in the section above, the single partitioned model for BizTalk Server development is generally recommended for medium to high complexity BizTalk projects. For this reason a master solution should be created that will hold all the subprojects. This master solution will typically be used as the main build solution. For notes on creating a master solution for a partitioned single solution in Visual SourceSafe see "Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution."
  • Store all Visual Studio .NET BizTalk Server 2004 projects in a folder under the master solution. If Visual SourceSafe is being used as the source control environment then it is necessary to ensure that all subprojects are created in a folder underneath the folder holding the master solution that contains them.
    ms942199.note(en-US,BTS.10).gifNote
    It is possible to store multiple solution files in the same folder, but this limits the ability of Visual SourceSafe to create partitioned solutions.

    For notes on adding subprojects to a master solution for a partitioned single solution in Visual SourceSafe see "Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution."
    For more information, see "Working with Visual SourceSafe" in the MSDN white paper Team Development with Visual Studio .NET and Visual SourceSafe that was referenced at the beginning of this section.
  • Divide the folder structure into shared and process-specific sections. When creating the file structure it is usual to divide up the folder structure to separate the shared entities versus the specific business process entities. The shared entities are common to multiple projects and may include helper classes.
    For example, the first three folders in the following list are organized by shared entity and the last two are organized by business process:
    C:\SysInteg2004\_Master_Solution\Shared\
    C:\SysInteg2004\_Master_Solution\Shared\EmailHelper
    C:\SysInteg2004\_Master_Solution\Shared\SharedSchema
    C:\SysInteg2004\_Master_Solution\AccountRequest\
    C:\SysInteg2004\_Master_Solution\AccountValidate\
    
  • Pipeline projects. During development of pipeline components, it is a common requirement to modify and retest a pipeline independent of the rest of the solution. For this reason it is often helpful to keep pipeline solutions as a separate BizTalk Server 2004 project, resulting in a separate pipeline assembly that can be removed and redeployed without the need to perform rebinding of the rest of the solution.
    Additionally it is common practice to keep the code that implements the actual pipeline interfaces and pipeline processing logic in a separate project with a reference from the BizTalk Server 2004 project (containing the .btp files) to the C# or Microsoft Visual Basic® .NET project.
    For notes on debugging pipelines see "Debugging Pipelines" later in this document.
  • Creating deployment and test scripts. When developing with BizTalk Server 2004 it is common to automate complex or often-repeated tasks by using scripts or command (.cmd) files. Nearly all BizTalk Server 2004 deployment and runtime tasks are exposed as command-line tools or Windows Management Instrumentation (WMI) interfaces to enable the development of scripts to aid automation of these tasks.
    To enable the development of a unified build and deployment process, build and install scripts should be built by the individual developers, but viewed as reusable parts of the complete solution. If the scripts are written to be reusable then tasks like the deployment of a complete solution package can reuse the scripts. For example, if the deployment and undeployment scripts accept parameters to specify the paths for the DLL and binding file folder locations, the scripts can be reused when the compiled BizTalk Server assemblies are in a different location.
    Using this approach, each developer adds the install scripts for their specific processes to the folder and then a single process can easily be written to perform a full deployment of all processes. For more information about deployment scripts see "Automating Deployment Tasks." For more information about build scripts see "Automated Build Processes."
    Scripts are typically stored along with the process they reference, using standard script names as in the following examples:
    C:\SysInteg2004\_Master_Solution\AccountValidate\scripts
    C:\SysInteg2004\_Master_Solution\AccountValidate\scripts\deploy.cmd
    C:\SysInteg2004\_Master_Solution\AccountValidate\scripts\undeploy.cmd
    
    In this way solution-wide scripts can be built up that call the individual process scripts (for example, SolutionDeploy.cmd calls AccountValidate\scripts\undeploy.cmd).
    An alternative approach may dictate that the scripts used to deploy a BizTalk Server 2004 solution need to be in a single shared folder. In this case the scripts need to be named according to the process they relate to. Again this approach can be used to build solution-wide scripts by calling individual scripts.
  • Strong name keys. All BizTalk Server 2004 projects result in assemblies that are deployed to the global assembly cache, and as such they must be signed with a strong named key during compilation.
    Typically a single Visual Studio .NET solution uses a single key file. This is true because the solution is treated as a single entity and is managed as such. If the business solution being developed is actually composed of two or more distinct parts, then consider if two key files should be used. Multiple keys in this scenario allow the parts to be treated as independent entities in the future, for example with differing upgrade paths.
    When considering helper projects, the same considerations apply. If the helper project is (or will be) a separate entity then it should be built using a separate strong name key.
    In an organization that has a closely guarded secure key that developers do not have access to, consider using delayed signing with the key. For more details on delayed signing, see Delay Signing an Assembly in the .NET Framework Developer's Guide available in the MSDN Library.
    The process of creating the key and including it in the project is simple; however developers should be aware that the path to the key stored in the BizTalk Server 2004 project is stored by default as an absolute path that includes the drive name (for example, Z:\CommonFiles\Keys). This has the disadvantage that projects cannot be edited or compiled on another workstation unless the path to the keys is exactly the same (or the key paths are modified every time the project is moved).
    If it is required to not tie the project to a specific drive name it is possible to change the path to a relative path (and hence remove the drive name) by modifying the assembly key file path.
    For a BizTalk project with the default folder structure for the output assembly, for example:
    <project folder>\bin\development 
    
    The key file path will be modified to the following:
    ..\..\<keyfilename>.snk 
    
    The following figure shows the relative key file path.
    Figure 4 Relative strong name key paths
    ms942199.Local_-716283125_figure04(en-US,BTS.10).gif

    It is often helpful to build a project directory structure that can be replicated upon every workstation with a specific path to the key in the structure, for example:
    C:\SysInteg2004\_Master_Solution\Shared\Keys
  • Test harnesses and stubs. It is generally recommended that test harnesses and test stubs be developed early in the project because they are useful resources and they also help develop a deeper understanding of the actual interactions between the integration layer and the other systems. If test harnesses are kept under the master solution then they can be included in the source control of a specific process or under a more general shared project. For example:
    C:\SysInteg2004\_Master_Solution\Shared\Test
    For more details on test harnesses, see "Testing BizTalk Server 2004 Solutions."
  • Cross-reference seed data. If the BizTalk Server 2004 cross-reference databases are being used as part of the business processes under development then it is often necessary to load the databases with the necessary data to allow lookup operations to successfully take place. Cross-reference data can be input using the BizTalk Server 2004 Cross Reference Import Tool (btsxrefimport.exe) and XML "seed files."
    When the import tool is run it empties the cross-reference databases before importing new data from the seed file. This means that prior to the development of the "all processes" deployment script all seed data must be consolidated into one seed file. For example:
    C:\SysInteg2004\_Master_Solution\Shared\CrossRefData\MasterSeedData.XML
    
    Early in the development cycle, individual developers may need to seed their local database with process-specific data only. In this case the seed data should be held in individual files, named according to the process they relate to. For example:
    C:\SysInteg2004\_Master_Solution \Shared\CrossRefData\AccountRequestSeedData.XML
    
  • Storing BizTalk binding files. When developing BizTalk Server 2004 projects binding files are produced that define the binding of a logical process to the physical transports and hosts. This binding information is initially created by using the GUI tools but is then persisted in binding files to ensure that a process can easily be deployed in a repeatable, scripted manner.
    Complexity arises from the fact that the physical binding information for the development environment may be different from that in the test, pre-production, and final production environments. Additionally the binding files contain references to the specific version numbers for the DLLs being deployed, and consequently these files need to be managed to be kept in sync with any DLL version number changes.
    To help manage this complexity the binding files should always be located within a specific location for a given process and named following a naming convention. This makes it easier to perform modifications to the binding files (either manually or automatically).
    Typically binding files should be kept in a folder called "bind" underneath the project folder and named according to a naming standard. For example, the binding files for the "AccountRequest" process in the development environment could be kept in a file as in the following example:
    C:\SysInteg2004\_Master_Solution \AccountRequest\bind\bind_AccountRequest_Stage1_DevEnv.xml
    
    Note how this convention allows binding files from multiple processes to be moved to a single folder location and still be identified. In some solutions multiple binding files may be required. Ensure that the naming convention used can take this into account.
  • Location for file dependency DLLs. When creating the folder structure it is helpful to create a common folder to hold DLLS that are referenced as file dependencies. By ensuring that all developers follow the same folder structure, file references will not become broken when solutions are moved between developers or workstations. For example:
    C:\SysInteg2004\ReferencedBin
    
  • Test messages. When developing schemas and using messages it is common to spend considerable effort testing many different sample messages against XML schema and orchestration processes.
    In a real-world solution, sample messages are a valuable asset because they contain data that is meaningful to the business solution. Random or dummy values in the same message will typically cause the process to fail. For this reason sample messages should be treated with as much care as the code of the solution.
    To assist in managing the message files, keep test messages under a specific folder called "msgs". In cases where there are both "generated instances" and "example messages" (for example, messages from an existing system) it is useful to keep these messages separate to allow comparison between the developed schema and actual data. For example:
    C:\SysInteg2004\_Master_Solution\AccountRequest\msgs\
    C:\SysInteg2004\_Master_Solution\AccountRequest\msgs\generated\
    C:\SysInteg2004\_Master_Solution\AccountRequest\msgs\examples\
    
    It is common in an integration project to have multiple versions of schemas and consequently, multiple versions of message files. In the same way that code has a version number applied, it is worth considering ensuring that version numbers are used when naming files.
    It may also be worth considering storing message instances within Visual SourceSafe. This has the added advantage of allowing comments to be added to versions and previous versions to be examined.
    ms942199.note(en-US,BTS.10).gifNote
    Instances of a message can be generated from a schema and can also be validated by using the "Generate Instance" function in the Visual Studio .NET BizTalk Server 2004 Schema editor. This function is extremely useful when testing a newly created schema against a system to be integrated. The generation capability also allows you to start development when the schema of a message is known but you have not yet been provided with instance messages.

  • Example folder structure. The example folder structure described above typically looks like the following for a sample project:
    C:\SysInteg2004\ReferencedBin\
    C:\SysInteg2004\_Master_Solution\Shared\Keys\
    C:\SysInteg2004\_Master_Solution\Shared\CrossRefData\
    C:\SysInteg2004\_Master_Solution\Shared\TestHarnesses\
    C:\SysInteg2004\_Master_Solution\Shared\SharedSchema\
    C:\SysInteg2004\_Master_Solution\Shared\EmailSendOrchestration\
    C:\SysInteg2004\_Master_Solution\Shared\ErrorHandlingOrchestration\
    C:\SysInteg2004\_Master_Solution\AccountRequest\
    C:\SysInteg2004\_Master_Solution\AccountRequest\bind\ C:\SysInteg2004\_Master_Solution\AccountRequest\msgs\
    C:\SysInteg2004\_Master_Solution\AccountRequest\msgs\generated\
    C:\SysInteg2004\_Master_Solution\AccountRequest\msgs\examples\
    C:\SysInteg2004\_Master_Solution\AccountRequest\scripts\
    C:\SysInteg2004\_Master_Solution\AccountRequest\AccReqRecPipe
    C:\SysInteg2004\_Master_Solution\AccountRequest\AccReqRecPipeCode
    

Working with BizTalk Server and Visual SourceSafe

This document assumes that the team developing the BizTalk Server 2004 solution consists of multiple developers who will be using Visual SourceSafe as their source control tool.

For step-by-step notes on creating a master solution for a partitioned single solution in Visual SourceSafe see "Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution."

Solution Character Sets

There is a known limitation with Visual SourceSafe related to the handling of Unicode files. When working with BizTalk solutions it is necessary to prevent Visual SourceSafe from corrupting BizTalk Unicode files by performing the following steps before adding a BizTalk Server 2004 solution to Visual SourceSafe:

  1. Start Visual SourceSafe 6.0 Admin.
  2. Select the SourceSafe database to be used.
  3. On the Tools menu, click Options.
  4. Click the File Types tab.
  5. Add the following to the end of the list of binary files. Verify there are semicolons between each file type.
    *.btm;*.btp;*.xsd;*.odx
  6. Click OK.

The preceding steps cause Visual SourceSafe not to inspect BizTalk files and attempt to change their formatting.

ms942199.note(en-US,BTS.10).gifNote
Files of the above type previously added to Visual SourceSafe will need to be deleted, including selecting the check box to destroy them permanently. Files then need to be re-added to Visual SourceSafe. Ensure that a backup of the files exists before carrying out this step.

Use Visual Studio .NET for Source Control Operations

All project creation and manipulation within Visual SourceSafe should be performed by using the integrated Visual SourceSafe support menus within Visual Studio .NET. Do not use Visual SourceSafe Explorer to perform these operations.

ms942199.note(en-US,BTS.10).gifNote
Even when using the Virtual PC approach to development, Visual SourceSafe should be used in exactly the same manner as in a traditional development environment.

When to Check In BizTalk Server 2004 Projects

The recommended approach to using Visual SourceSafe is to check in code only when it has successfully passed functional tests and the developer is confident that the code will successfully build without breaking any related code. Applying this model to BizTalk Server 2004 results in the following guidelines:

  • BizTalk Server 2004 projects that contain only message schema should not be checked in until the schema have been successfully tested against a variety of sample messages.
  • BizTalk Server 2004 projects that contain a business process should not be checked in until the solution has been successfully tested using the appropriate input and output messages using the correct send and receive ports.
  • ASP.NET Web service projects should not be checked in until the Web service code has been tested against the initiating system or by using a test harness.

If this model is followed, then the Visual SourceSafe repository will always hold a build that can be successfully built and tested. This principle is important if the approach of "nightly builds" is to be adhered to.

Checking In Intermediate Versions

An alternative approach to check-in is that of checking in "intermediate" versions. In this approach an intermediate version will not yet have successfully passed functional tests and can be thought of as "between builds." This is generally not a recommended approach because it breaks the general principle of always having a buildable version within the source control system. However some teams prefer this approach because it allows developers to use the source control system to check in and roll back versions without needing to fulfill the criteria needed to check in a formal build.

If the approach of checking in intermediate versions is required, then the source control can no longer be assumed to contain a "buildable" version. Instead it is required to distinguish between the intermediate versions and build versions. Using Visual SourceSafe this can be done in a variety of ways, either automatic or process based. For example:

  • Developers follow a process of notifying the build manager when a "buildable" version is added to Visual SourceSafe.
  • Developers check in a tested version, ready for building, into a "promoted area" within Visual SourceSafe. This promoted area is then used as the source upon which the master build is based.
  • Code or script can be written that uses the Visual SourceSafe COM interfaces. For example, a specific label can be used to denote code ready for a build and the script searches for this label and then "pins" this source into a separate Visual SourceSafe branch This branch is then used as the source for master builds.

Version Controlling Non-BizTalk Server Project Files

BizTalk Server uses additional files that can beneficially be versioned and stored in Visual SourceSafe. The following files are examples:

  • Binding files (both development and test)
  • Custom pipeline assemblies
  • Test data (for example, test messages)
  • Test harnesses (which may change over the project lifetime)
  • Build, deployment, and start-and-stop scripts that may need to be shared between development and build teams

If these files are related to a specific Visual Studio .NET BizTalk project then these files can be included within the BizTalk project and managed by using the Visual Studio .NET integrated source control functions. To include a file or folder into an existing Visual Studio .NET project, do the following:

  1. In Solution Explorer, click Show All Files.
  2. Select the folder or file to include in the solution.
  3. Right-click the folder or file, and then select Include In Project.

The following figure shows the screen for including files in a project.

Figure 5 Including files in a project

n/a

If the non-BizTalk Server project files are not part of a Visual Studio .NET project, then you can manage them by using Visual SourceSafe Explorer.

ms942199.note(en-US,BTS.10).gifNote
Visual SourceSafe Explorer should NOT be used to manage any items that are part of a Visual Studio .NET project under source control.

Checking the Visual SourceSafe Database

When Visual SourceSafe databases are large, heavily used, or being accessed by many concurrent users, it is recommended to regularly run the "Analyze VSS DB" command from the Visual SourceSafe menu. If the analysis detects errors, then they can be fixed by using the "Analyze and Fix VSS DB" command. Both these commands need to lock the Visual SourceSafe database, and hence require everyone to be disconnected from Visual SourceSafe.

Creating an Example Solution Structure and Integrating with Visual SourceSafe

"Appendix: Step-by-Step Guide to Setting Up a Partitioned Single Solution" contains a step-by-step guide to creating a partitioned single solution structure using Visual Studio .NET and Visual SourceSafe that follows the guidance listed earlier. This step-by-step guide is intended as a working example and individual projects should modify the steps listed to meet their requirements.

Working with BizTalk Server and Assembly Version Numbers

When developing with the .NET Framework, versioning is governed by a standard set of rules that ensure that when a version number changes, the impact of that change is typically minimal. Due to certain design and implementation choices made during the development of BizTalk Server 2004, version numbers with BizTalk Server 2004 do not always follows the standard .NET Framework rules. The following sections describe these conditions.

Overview of .NET Versioning

When a BizTalk Server 2004 project is compiled, the resulting DLL is a .NET assembly and its behavior follows the standard .NET versioning behavior. If it occurs that multiple versions of the same assembly are installed with the same assembly version number, the system will produce unexpected results. For this reason it is important to ensure that BizTalk Server 2004 assembly versioning is planned and managed.

Each DLL containing a .NET assembly has two distinct ways of expressing version information. The following table shows the difference between the assembly version and the file version.

Table 10 DLL version numbers

Assembly version File version

The assembly version number together with the assembly name and culture information is part of the assembly's identity. The assembly version number is used by the runtime to enforce version policy and plays a key part in the type resolution process at run time. This version number is physically represented as a four-part string with the following format:

<major version>.<minor version>.<build number>.<revision>

The file version is a string that is displayed by Microsoft Windows® Explorer. This number can be used to manually determine which version of a file is installed.

The following figure shows where these two version numbers can be viewed for a DLL.

Figure 6 DLL version numbers

n/a
Implications of Changing Version Numbers

In .NET development it is typical to update the assembly version number to the current build number when a build takes place. However when developing a BizTalk Server 2004 solution, changing the assembly version number can break the relationship between an assembly and the dependent items that reference the DLL by its assembly version number. The following table lists items that refer to a BizTalk Server 2004 assembly by using its version number and the effect of changing an assembly version number.

Table 11 Entities affected by assembly version number changes

Entity Effect of changing assembly version number

Binding files

Changing the assembly version number causes any existing binding files that reference the assembly to fail. This is because the binding file references the assembly by attributes including its version number. To reuse existing files they will need to be modified or regenerated. Binding files are XML and so modification can be undertaken using tools like Visual Studio .NET, Microsoft Office Infopath®, or Notepad, or alternatively can be scripted or automated by using a tool.

See "Patching Binding Files" for more information about reusing existing binding files using tools.

See "BizTalk Server 2004 Deployment Toolkit" later in this document for examples of scripted binding file modification as part of a scripted build process.

BAM tracking profile definition file (.btt) files

Changing the assembly version number causes any existing BAM tracking profile definition file files to fail. The BAM tracking files are a binary file format so they cannot be edited and instead must be regenerated. If BAM tracking profiles are required it may be necessary to do either of the following:

  • Avoid frequently updating version numbers during the build process.
  • Delay building BAM tracking profiles until version numbers are stable.

Web services published by using the Web Services Publishing Wizard

When the Web Services Publishing Wizard is used to produce an ASP.NET Web interface, the assembly version of the BizTalk Server DLL is included in the ASP.NET source code. The assembly version number is used at runtime by the ASP.NET interface as part of the bodyTypeAssemblyQualifiedName property of the Web service operation. If the version number of the BizTalk Server assembly changes without updating the bodyTypeAssemblyQualifiedName property then subsequent Web service operations will be rejected by BizTalk Server.

If the receive location uses the xmldefaultpipeline the subscription relies on the document type and will use the embedded assembly information and fail if the assembly does not exist. If you use the passthrupipeline (which is the default if you expose a port and let the wizard create the receive location) it ignores this embedded assembly information.

Approaches to Incrementing BizTalk Server 2004 Assembly Version Numbers

During a project you have a choice between the following:

  • Choose a fixed assembly version for a given deliverable and increment only the file version number.
  • Increment both the assembly version and the file version as the development progresses.

The following table compares these two possible approaches to updating the version numbers.

Table 12 Comparing approaches to update version numbers

Increment the informational file version only and keep a fixed assembly version Increment both the assembly version and the informational file version

Assembly version number = Fixed number

File version number = Build number

Assembly version number = Build number

File version number = Build number

BizTalk Server runtime may pick up wrong version of assembly if multiple assemblies installed

BizTalk Server always runs latest version of assembly, even if multiple assemblies installed

Only one version of solution can be deployed at any time

Can deploy different versions of the solution side by side (although other aspects of the solution may clash, for example, schema definitions)

BizTalk Host needs to be restarted to force the loading of updated assemblies

Forces BizTalk Server to load new assemblies

Less work to create a new deployment because files that reference the assembly version number (for example, binding files and tracking profiles) do not need to be edited

More work for deployment because files that reference the assembly version number need to be kept updated with new version

When to Change File and Assembly Versions

Creating a policy on when to increment a BizTalk Server 2004 assembly version number depends on several variables including build processes, testing processes, number of assemblies and binding files, and the type of build being delivered.

The following table compares shipping and non-shipping build types.

Table 13 Comparing build types

Build type: Non-shipping Build type: Shipping

Non-shipping builds are builds that are not intended to ship to an end user. Non-shipping builds are the builds typically produced during the development and test cycle and are used only by the development and test teams. Non-shipping builds are typically unsupported.

Shipping builds are intended to ship to an end user and become deployed on the end user's systems. Shipping builds are supported.

Approach 1: Fix assembly versions for non-shipping builds

Following this approach, assembly versions are kept fixed and file version numbers are updated with each build, as shown in the following table.

Table 14 Build types for Approach 1

Build type: Non-shipping Build type: Shipping

Increment the informational file version only and keep a fixed assembly version.

Increment both assembly version and informational file version.

Following this approach means that binding files do not need to change; however the following additional requirements should be noted:

  • Ensure that the file version is always modified to reflect the build number. If the file version is not modified then it will become much more difficult to distinguish between different assembly versions. Relying on other attributes like file size and date stamps is less accurate.
  • Note that previously deployed BizTalk Server 2004 assemblies will be overwritten by the next deployment, so parallel deployments of versions will not be possible.
  • Because the assembly version number is not changing, BizTalk Server 2004 will not detect that the assembly has changed and so it will be necessary to force the BizTalk Server 2004 runtime to load the new version into the host's memory. This can be achieved by stopping the BizTalk Server 2004 runtime. For more details on how to achieve this, see "Restarting BizTalk Server Hosts and Services."
  • It can be advantageous to have a process output its current build number as debug information. This ensures that when processes run, it is possible to confirm that all the build versions are as expected. See "Output Assembly Version to Aid Tracing, Debugging, or Exception Handling" for more details on how to view version number information at run time.

Approach 2: Increment assembly versions for non-shipping builds

Following this approach, assembly version numbers are incremented for every build, including non-shipping builds, as shown in the following table.

Table 15 Build types for Approach 2

Build type: Non-shipping Build type: Shipping

Increment both assembly version and file version.

Increment both assembly version and file version.

This approach incurs the added effort of modifying the assembly build numbers and modifying the associated dependencies that use the build numbers (like binding files).

If the BizTalk Server 2004 development team follows the approach of incrementing assembly version numbers for every build, then there are several possible ways of automating the required changes in version numbers and binding files. These options are discussed in "Approaches to Incrementing BizTalk Server 2004 Assembly Version Numbers."

Which approach versioning should you use? For the majority of developments Approach 1 (fixed assembly versions for non-shipping builds) is typically suitable. It has the advantage of requiring fewer resources to test and as long as informational version numbers are kept updated, DLL conflict issues should be detectable and resolvable by using the informational version number.

Map Function Version Numbering

.NET assemblies can be invoked from within a map (using the scripting functoid, found under the advanced functoids palette), and this provides a great deal of flexibility for delivering custom map functionality. However it is important to understand that the internal representation of the map files references not only the assembly type name but the full assembly version number.

This is significant because it means that if the version number of the assembly called by the map changes, then any links that reference the assembly will break. In a complex map this can cause considerable rework.

To avoid this issue it is recommended that if assemblies are required to be called from a map, a specific assembly is created to hold only map functionality and the assembly version number of this assembly is fixed. In this way, other helper functions can have the assembly version updated without breaking the maps.

If an assembly referenced from a map is changed after map development then consider updating the map file outside of the Visual Studio .NET BizTalk Server 2004 Map Editor to reflect the updated version numbers.

To achieve this use the following steps:

  • Do not open the map by double-clicking it (because this will cause the links to break and incur rework).
  • Open the map by using the Visual Studio .NET XML editor: Right-click the map in Solution Explorer, click "Open with", and then choose "HTML/ XML editor".
  • After the instances of the "Map only" assembly reference number have been located in the XML structure they can be replaced with the updated version number by using "Replace" on the Edit menu.
  • The map can then be saved, closed, and opened by using the standard BizTalk Server 2004 map editor.

The version number of helper classes referenced by project references or file reference and used by orchestrations (for example, from within expressions) can be changed without encountering the issue described above.

Developing BizTalk Server 2004 Orchestration Designs

After the completion of the information-gathering phase, described in the first section of this document, it is usual for the team to begin to produce design documents for the BizTalk Server 2004 orchestrations. These documents are typically then used to validate the proposed designs with system and process owners prior to commencing the development phase.

One of the primary benefits of the BizTalk Orchestration model is the great clarity you can achieve when a software implementation is pictorial. Regardless of how well a developer comments code, there is always a need to maintain a separate set of artifacts including process diagrams, Visio diagrams (with varying shape usage), prose documents, and whiteboard discussions that are used to convey what the code is actually doing. This is especially true when working with a business audience.

When working with a BizTalk Orchestration, the diagram is the implementation of a piece of functionality (at least at a specific level). This provides opportunities to use an orchestration diagram in several helpful ways within a project life cycle, as follows:

  • To capture an initial high-level design, using all the orchestration shapes as intended but without fully implemented maps and schemas. These designs are often called "skeleton orchestrations." All of the external system interactions, communication patterns, decision points, parallel vs. joined flows, and so on, can be represented at this point in a skeleton orchestration.
  • To gain consensus with the development team and business sponsor about the functionality. An efficient route to discussion can be to place the high-level skeleton orchestration designs on a wall with project stakeholders and then walk through the processes. The Orchestration Designer for Business Analysts provides another way to walk through the functionality. This tool has the added benefit of allowing you to exclude what you might consider to be lower-level detail by setting the Report to Analyst switch on various orchestration shapes to False. (See "Diagramming Business Processes" earlier in this document.)
  • To estimate the number of tasks required and the level of effort needed to develop the solution. The various messages and shapes in your initial orchestration can often provide a reasonably granular approach for time estimating. If the complexity of messages is taken into account it is possible to construct a draft project plan that includes orchestration entities. If this draft plan is tested early on in the project, the approach can be refined to provide a reliable model for estimation.

Skeleton Orchestration Design

One approach to developing BizTalk Server orchestration designs is to use Visual Studio .NET to produce "skeleton" projects. The skeleton projects provide a way of validating the assumptions about the number and type of messages and the operations taking place upon the messages to complete a process.

These skeleton projects exhibit the following attributes:

  • Projects should be divided up in accordance with the planned schema and process ownership.
  • Projects contain skeleton schema with correct schema names. These schema contain correctly named root nodes and correct target namespaces, but need not contain any further schema detail.
  • Projects contain skeleton maps that have a source and destination schema but need not have any links.
  • Orchestrations reference the appropriate skeleton schema.
  • Orchestrations contain messages and variables created from the skeleton schema. This can be useful to understand which orchestrations share messages.
  • Orchestrations contain the receive and send operations dictated by the design.
  • Orchestrations contain the process logic associated with the process, including send, receive, mapping, and logic operations.
  • Expression shapes contain comments that describe the functionality of the expression. The expression shapes also need to contain at least one valid line of expression to allow the skeleton solution to compile without errors (for example, System.Diagnostics.Debug.WriteLine("In expression 1");).

As the designs are progressed, the skeleton projects can be used as the starting point for the BizTalk Server 2004 development phase. To add documentation to a group of related workflow shapes, use a Group shape. Group shapes display as much text as you care to associate with the shape. Many of the other orchestration shapes can hold descriptions, and it can be useful to use this to express planned implementation detail.

To make best use of the BizTalk Server skeleton designs, it is helpful to use a documentation tool to export the skeleton design. See "Documenting BizTalk Server 2004 Systems" later in this document.

BizTalk Server 2004 Naming Conventions

This section provides guidelines on naming projects, assemblies, orchestrations, messages, and other artifacts within BizTalk Server 2004.

XML Namespace Conventions

Prior to starting a BizTalk Server 2004 development it is good practice to ensure that a standard is created for new XML target namespaces. The actual standard is often less important than the uniformity it confers across the project. An example target namespace naming policy might be:

http://<organisationname>.com/namespaces/<system>/<operation>/<direction>

Where:

  • <organizationname> is the name of the organization as used in the organization's domain name
  • <system> is the business system that the message relates to
  • <operation> is the operation on the interface
  • <direction> is the direction of the message from the point of view of the system referred to by <system>
  • <direction> is typically "incoming" or "outgoing" for unidirectional messages
  • <direction> is typically "IncomingRequest", "OutgoingResponse", and so on, for bidirectional and synchronous systems
  • Optionally a year, date, or version number can be used as part of the message standard. However be aware that changing these will result in rework for the BizTalk Server developers because the maps and orchestrations will need to be updated to point to the "new" schemas. This can be a significant cost in a large solution.

BizTalk Artifact Namespaces

Many artifacts within a BizTalk solution have a standard .NET namespace associated with them. The MSDN paper "Standard guidance on .Net namespace naming" should be adhered to with BizTalk artifacts, namely:

CompanyName.TechnologyName.Feature

Note that these are Pascal-cased and nested namespaces will have dependencies on types in the containing namespace.

For more information about naming guidelines, see the Namespace Naming Guidelines article on the MSDN Web site at http://go.microsoft.com/FWLink/?LinkID=42405.

BizTalk Projects and Assemblies

BizTalk project names and assembly names should often match the name of the associated namespace, such as the following name:

CompanyName.projectName.Function.dll

A division into assemblies such as the following will often be quite suitable for a BizTalk project:

MyCompany.MyProject.OrderProcessingProcessOrch.dll

MyCompany.MyProject.InvoiceBatchProcess.dll

MyCompany.MyProject.SharedSchemas.dll

MyCompany.MyProject.Pipelines.dll

MyCompany.MyProject.PipelineAssemblys.dll

Orchestration Naming Conventions

When deployed, an orchestration "type name" is the name that is used and displayed by the BizTalk Server administration tools to refer to the orchestration. When creating a new orchestration, this property defaults to "orchestration_1". When creating a new orchestration, change this property to a descriptive name to help distinguish the orchestration.

Messaging Artifact Naming

All artifacts should be named with a Pascal convention unless mentioned otherwise for the specific artifact, although underscores are used to separate logical entities. For schemas, maps, orchestrations, and pipelines, ensure that the .NET type name matches the file name (without file extension).

The following table shows an example messaging artifact naming convention.

Table 16 Example messaging artifact naming convention

Artifact Standard Notes and examples

Schema file

<RootNode>_<Standard>.xsd or

<DescriptiveName>_<Standard>

Standards include XML, X12, FlatFile (FF), and other custom formats.

If root node does not distinguish the schema, use descriptive name.

Example:

PurchaseOrderAcknowledge_FF.xsd or

FNMA100330_FF.xsd

Property Schema file

<PropSchema>_<Standard>.xsd

Should be named to reflect its common usage, across multiple schemas if appropriate. Prefixed with "Prop_" to help distinguish it from standard message schema.

Example:

Prop_PONumber_XML.xsd

Map file

<SourceSchema>_To_<DestinationSchema>.btm

If XSLT file is specified for map, XSLT file should be named identically with .xsl extension. If orchestration contains multiple instances of messages derived from the SourceSchema, use the message instance names instead.

Example:

PurchaseOrder_FF_To_PurchaseOrderAcknowledge

Orchestration file

A meaningful name that represents the underlying process.

Example:

EvaluateCredit.odx

Send/Receive Pipeline file

<AppName>_Rcv_<SchemaName> or

<AppName>_Rcv_<Function>

<AppName>_Snd_<SchemaName> or

<AppName>_Snd_<Function>

A pipeline might be used to ensure reception of particular schema(s), or to perform some other function. "AppName" prefix (corresponding to name of the application deploying to BizTalk) helps when many applications are deployed to the same BizTalk installation.

Example:

ERP_Rcv_ PurchaseOrderAcknowledge_FF

ERP_Rcv_CatalogUnzip

Receive ports

<AppName>_<InputSchema>_To_<OutputSchema>

or

<AppName>_<FunctionalDescription>

"AppName" prefix (corresponding to name of the application deploying to BizTalk) helps when many applications are deployed to the same BizTalk installation.

Use functional description if the input schema (and potentially output schema, if request/response) do not adequately describe the port.

(No need to add Snd/Rcv/Port, etc. since they are grouped accordingly in admin tools.)

Examples:

ERP_PurchaseOrder_XML_To_POAck_XML

(for request/response port)

ERP_PurchaseOrder_XML

(for one-way port)

ERP_CheckOrderStatus

Receive locations

<AppName>_<ReceivePortName>_<Transport>

Example:

ERP_CheckOrderStatus_MSMQT

Send port groups

<AppName>_<FunctionalDescription>

Example:

CRM_CustomerUpdateNotification

Send ports

<AppName>_<Schema>_<Transport>

or

<AppName>_<FuncDescription>_<DestApp>_

<Transport>

In some cases, the schema being sent is descriptive enough. In others, a functional description of the action to be taken by a destination application is better suited.

Example:

CRM_CustomerUpdate_ERP_MSMQT

Parties

A meaningful name for a trading partner.

If dealing with multiple entities within a trading partner organization, the organization name could be used as a prefix.

Roles

A meaningful name for the role that a trading partner plays.

Example:

Shippers

Note that the .NET type name of the artifacts listed above should match (without the file extension) the file name. The .NET namespace will likely match the assembly name.

Orchestration Shape Naming

Establishing and following naming conventions are good practices for designating variables, messages, multipart types, and so on, but they become even more important for the workflow shapes contained within an orchestration. The goal is to ensure that the intent of each shape is clear, and that the text associated with the shape conveys as much as possible given the space constraints. In this way, a non-technical audience will be able to use the orchestration as documentation.

ms942199.note(en-US,BTS.10).gifNote
To add documentation to a group of related workflow shapes, use a Group shape. Group shapes display as much text as you care to associate with them, and can add quite a bit of documentation value to the diagram. However, if you intend to instrument the application by using BAM, be aware that a Group shape does not exist at run time and as such cannot be part of a BAM tracking profile. If this is the case, you could use a non-transactional scope shape instead.

The following table shows an example orchestration shape naming convention.

Table 17 Example orchestration shape naming convention

Shape Standard Notes and examples

Scope

Scope_<DescriptionOfContainedWork> or

Scope_<DescOfcontainedWork>_<TxType>

Including brief information about transaction type may be appropriate. Example:

Scope_CreditServiceCall

Receive

Rcv_<MessageName>

Typically, MessageName will be the same (though Pascal-cased) as the name of the message variable that is being received "into" (though the message variable will be camel-cased).

Example:

Rcv_RawCreditReport

Send

Snd_<MessageName>

Typically, MessageName will be the same (though Pascal-cased) as the name of the message variable that is being sent (though the message variable will be camel-cased).

Example:

Snd_POAcknowledge

Expression

<DescriptionOfEffect>

Expression shapes should be named with Pascal convention (no prefix) to simply describe the net effect of the expression, similar to naming a method.

Example:

GetFindingsReport

Decide

Decide_<DescriptionOfDecision>

Decide shapes should be prefixed with "Decide_" followed by a full description of what will be decided in the "if" branch.

Example:

Decide_ApprovalRequired

If-Branch

If_<DescriptionOfDecision>

If-branch shapes should be prefixed with "If_" followed by a (perhaps abbreviated) description of what is being decided.

Example:

If_ApprovalRequired

Else-Branch

Else

Else-branch shapes should always be named "Else".

Example:

Else

Construct Message (Assign)

Assign_<Message> (for Construct)

<ExpressionDescription> (for expression)

If a Construct Message shape contains a message assignment, it should be prefixed with "Assign_" followed by an abbreviated name of the message being assigned.

The actual message assignment shape contained should be named to describe the expression that is contained.

Example:

Assign_PaymentVoucher

which contains expression:

CopyPaymentDetails

Construct Message (Transform)

Xform_<SourceSchema>To<DestSchema> (for Construct)

X_<SourceSchema>To<DestSchema>

(for expression)

If a Construct Message shape contains a message transform, it should be prefixed with "Xform_" followed by an abbreviated description of the transform (that is, source schema to destination schema).

The actual message transform shape contained should generally be named the same as the containing shape, except with an "X_" prefix to save space ("X_LoanRequestToCreditRequest").

Example:

Xform_LoanRequestToCreditRequest

which contains transform shape:

X_LoanRequestToCreditRequest

Construct Message

(containing multiple shapes)

N/A

If a Construct Message shape uses multiple assignments or transforms, the overall shape should be named to communicate the net effect, using no prefix.

Call/Start Orchestration

Call_<OrchestrationName>

Start_<OrchestrationName>

N/A

Throw

Throw_<ExceptionType>

The corresponding variable name for the exception type should (often) be the same name as the exception type, only camel-cased.

Example:

Throw_RuleException, which references the "ruleException" variable.

Parallel

Parallel_<DescriptionOfParallelWork>

Parallel shapes should be named "Parallel_" followed by a description of what work will be done in parallel.

Example:

Parallel_CreditVendorCalls

Delay

Delay_<DescriptionOfWhatWaitingFor>

Delay shapes should be named "Delay_" followed by an abbreviated description of what is being waited for.

Example:

Delay_POAcknowledgeTimeout

Listen

Listen_<DescriptionOfOutcomes>

Listen shapes should be named "Listen_" followed by an abbreviated description that captures (to the degree possible) all the branches of the Listen shape.

Example:

Listen_POAckOrTimeout Listen_FirstShippingBid

Loop

Loop_<ExitCondition>

Loop shapes should be named "Loop_" followed by an abbreviated description of what the exit condition is.

Example:

Loop_AllMsgsSent

Role Link

N/A

See "Roles" in Table 16 above.

Suspend

Suspend_<ReasonDescription>

Describe what action an administrator must take to resume the orchestration. More detail can be passed to error property, and should include what should be done by the administrator before resuming the orchestration.

Example:

Suspend_ReEstablishCreditLink

Terminate

Terminate_<ReasonDescription>

Describe why the orchestration terminated. More detail can be passed to error property.

Example:

Terminate_TimeoutsExpired

Call Rules

CallRules_<PolicyName>

The policy name may need to be abbreviated.

Example:

CallRules_CreditApproval

Compensate

Compensate

or

Compensate_<TxName>

If the shape compensates nested transactions, names should be suffixed with the name of the nested transaction. Otherwise it should simply be Compensate.

Example:

Compensate_TransferFunds

or

Comp_TransferFunds

For documentation purposes, we recommend that developers add descriptive text to the shapes' description property. Third party BizTalk documenter tools (see "Documenting BizTalk Server 2004 Systems" below, can generate Microsoft Word documents or a compiled help file that contain these shape descriptions. These documents are used as the basis of system documentation.

Orchestration Type Naming

The following table shows an example orchestration type naming convention.

Table 18 Example orchestration type naming

Shape Standard Notes and Examples

Multi-Part Message Types

<LogicalDocumentType>

Multi-part types encapsulate multiple parts. The WSDL specification indicates "parts are a flexible mechanism for describing the logical abstract content of a message." The name of the multi-part type should correspond to the "logical" document type, that is, what the sum of the parts describes.

Example:

InvoiceReceipt

(which might encapsulate an invoice acknowledgment and a payment voucher)

Multi-Part Message Parts

<SchemaNameOfPart>

Should be named simply for the schema (or simple type) associated with the part.

Example:

InvoiceHeader

Messages

camelCased

See "Message Instance Naming" below.

Variables

camelCased

Example:

debugString

Port Types

<function>

Should be named to suggest the nature of an endpoint with Pascal casing. If the orchestration is exposed as a Web service, the port name is exposed in the generated WSDL. For this reason these names are not suffixed with "PortType" to avoid the "PortType" being visible to external interface.

If there will be more than one Port for a Port Type, the Port Type should be named according to the abstract service supplied.

Example:

ProcessPurchaseOrderPortType

which might have operations such as

SubmitPO
RequestPOStatus

Ports

<function>Port

Should be named to suggest a grouping of functionality, with Pascal casing and suffixed with "Port." Typically, the port name should reflect the likely name of the physical port created in the binding files. This will aid configuration.

Example:

ProcessPurchaseOrderPort

Correlation types

pascalCased

Should be named with Pascal-case convention, based on the logical name of what is being used to correlate.

Example:

PurchaseOrderNumber

Correlation sets

camelCased

Should be named with camel-case convention based on the corresponding correlation type. If there is more than one, it should be named to reflect its specific purpose within the orchestration.

Example:

purchaseOrderNumber

Orchestration parameters

camelCased

Should be named with camel-case convention, and match the caller's names for the corresponding variables where appropriate.

Message Instance Naming

When naming messages within orchestrations, you should use a standard naming convention to avoid the confusion that can arise when messages are traveling between multiple systems in both directions.

Consider naming messages from the point of view of the integration layer (that is, incoming means incoming to the integration layer, outgoing means leaving the integration layer). It may help to include underscores (_) in the names.

The following table shows an example message naming convention.

Table 19 Example message naming convention

Message name Usage
<SystemName><OperationName>Incoming

To describe a message received from an asynchronous system by the integration layer

<SystemName><OperationName>Outgoing

To describe a message being sent to an asynchronous system by the integration layer

<SystemName><OperationName>OutRequest

To describe the outgoing request message sent to a synchronous system by the integration layer

<SystemName><OperationName>OutResponse

To describe the associated response incoming message received from a synchronous system by the integration layer

<SystemName><OperationName>InRequest

To describe the incoming request message received from a synchronous system by the integration layer

<SystemName><OperationName>InResponse

To describe the associated response outgoing message sent to a synchronous system by the integration layer

If the messages being described are traveling between multiple possible systems, then a naming convention based upon system names may not be meaningful. An alternative approach is to name the messages after their schema, with a description of the message usage in the orchestration, for example:

CRMCustomerUpdate_validate_Incoming
CRMCustomerUpdate_newAddress_Outgoing

Documenting BizTalk Server 2004 Systems

The configuration of a deployed BizTalk Server 2004 system is stored within the configuration database of the BizTalk Server 2004 group. Tools have been written to query the Configuration database and produce formatted output that lists the configuration information to help you document BizTalk Server 2004 systems.

These tools can be useful for:

  • Documenting a skeleton design early in the design process
  • Producing complete system documentation
  • Validating a newly deployed system against an expected system configuration

You can find a BizTalk Server 2004 reporting tool that generates a compiled help file detailing the BizTalk Server configuration. For more information, see Configuration Management Power Toys at the BizTalk ChalkTalk Web blog at http://go.microsoft.com/fwlink/?LinkId=43658.

This tool creates compiled help (.chm) and Word 2003 XML files to quickly document the many artifacts (hosts, ports, orchestration diagrams, schemas, maps, pipelines, adapters, rule engine vocabularies and policies, and more) and their dependencies within a BizTalk Server 2004 environment.

ms942199.note(en-US,BTS.10).gifNote
This tool may be moved to MSDN in the future, so if the link above is not available, search MSDN for "BizTalk 2004 Documenter."

Testing BizTalk Server 2004 Solutions

By the nature of integration development, testing BizTalk Server 2004 solutions involves exchanging data with other systems. To functionally test BizTalk Server 2004 processes, you must obtain suitable test data and have access to the appropriate method of interacting with this data. In some cases you can interact with the actual systems that the integration layer requires to exchange data with during the development stage. In many cases, however, this will not be possible. In this scenario it is typical to use test harnesses and test stubs to allow development to proceed in the absence of the actual systems.

Test Data

Test data is a very significant resource when developing an integration solution. Without valid test data you often cannot test a business process fully. When planning an integration development, you must ensure that the project includes sufficient resources and time to obtain valid data to allow testing. This is particularly true when the test data needs to be produced from systems that have never previously been integrated.

If existing data flows already exist it can be extremely beneficial to the design and testing of the integration solution if example messages are captured for all existing business operations.

Test Harnesses and Test Stubs

For integration environments, test harnesses and test stubs can be defined as the following:

  • Test harness. Code or utilities used to allow the testing of an integration process. Typically a test harness is used to submit data to the integration layer to initiate a process.
  • Test stub. Code or utilities used to simulate a system that is not available, but from which a response is required to allow the testing of a process. Typically a test stub is used to accept data as if it was the unavailable system and respond with the appropriate response, thereby allowing the process to continue despite the absence of a system.

Example test harnesses include:

  • A Visual Studio .NET project simulating an unavailable system by making a Web service request into the integration layer using the WSDL "contract" file that describes the final SOAP calls between the system and the integration layer.
  • An ASP.NET application simulating an unavailable system by performing an HTTP POST to the integration layer, passing a sample message and displaying the responding XML output and HTTP response code.
  • Custom code that uses the System.Xml.XmlDocument classes to produce a specified number of test input XML files with varying values and sizes to allow the volume testing of a batch business process.

Example test stubs include:

  • A Visual Studio .NET project that simulates an unavailable system by accepting incoming Web service requests from the integration layer and returning a response. The project will potentially use some simple logic to vary the response values for testing purposes. The Web services WSDL may be provided from a system that already exists or that is to be built using the WSDL as the agreed interface.
  • An ASP.NET application that accepts HTTP POSTS from the integration layer and returns one of several possible XML messages based upon some simple logic.

It is generally recommended that test harnesses and test stubs are developed early in the project because they are a useful resource and they also help develop a deeper understanding of the actual interactions between the integration layer and the other systems.

For more information about example test stubs and harnesses, see "Useful Development Tools" later in this document.

This section provides information that may be useful to a team in the development phase of a BizTalk Server 2004 integration project. The first task you must complete is setting up your BizTalk Server 2004 development environment. This section provides notes on typical tasks required to set up a development environment.

Setting Up HTTP Receive

By default, Microsoft Windows Server™ 2003 installs in a locked-down configuration to reduce the possibility of security attacks being successful. However, this configuration does require additional steps over and above previous releases of the operating system. This is particularly true when configuring IIS, and for this reason the following example procedures are included.

The following procedure assumes the default BizTalk Server 2004 hosts have been created. For a solution that uses multiple hosts, the steps will have to be modified to reflect the choice of possible hosts.

  1. Using the Internet Information Systems (IIS) 6.0 Management Console (MMC), create a new application pool called BTSAppPool.
  2. When you create the application pool, set the application pool identity to the BizTalk Server 2004 Isolated Host User.
  3. Make sure the application pool identity password field is set. (Note that the user is not prompted to set the password.)
  4. Using the IIS 6.0 MMC, create a new virtual directory called NewHTTPRec that points to c:\Progam Files\Microsoft BizTalk Server 2004\HTTPReceive.
  5. Enable executable permission on the new virtual directory.
  6. Set the virtual directory application pool setting to use the newly created application pool (for example, BTSAppPool). This ensures that the virtual directory is running using the identity of the Isolated Host User. This user is required because it has the appropriate permissions to be able to interact with the BizTalk Server MessageBox.
  7. As a test, browse to the HTTP receive location (for example, enter the following address into a browser: http://127.0.0.1/NewHTTPRec/BTSHTTPReceive.dll). This should generate a "404 page not found" error on a clean IIS 6 configuration. This is because the running of the unknown ISAPI DLL (BTSHTTPReceive.DLL) is prohibited by default. If you get a different error, see steps 3 and 4 in the following procedure.

To allow the running of the BTSHTTPReceive DLL, complete the following steps:

  1. Using the IIS 6.0 MMC, select Web Service Extensions and click Add a new Web Service Extension.
  2. Call the extension HTTP Rec. Browse to the c:\Program Files\Microsoft BizTalk Server 2004\HTTPReceive\BTSHTTPRceive.dll and select the check box marked Set extension status to allowed. The following figure shows the New Web Service Extension dialog box.
    Figure 7 New Web Service Extension dialog box
    n/a

  3. Browse to the HTTP receive DLL (for example, enter the following address into a browser: http:/127.0.0.1/NewHTTPRec/BTSHTTPReceive.dll). This should generate a blank HTTP response and entries in the application event log with text similar to: The Messaging Engine failed to register the adapter for "BizTalk HTTP Receiver" for the receive location "/BTSHTTP/BTSHTTPReceive.dll". This event message is expected because the URL used to submit a message has not yet been set up as a receive location in BizTalk Server so the runtime does not yet know how to handle incoming data.
  4. Using the BizTalk Server 2004 administrator, create a new receive location using the following address /NewHTTPRec/BTSHTTPReceive.dll.
  5. When the browse test is carried out again, the application event log should contain an error message reporting that the HTTP receive function is not submitting a valid message to the BizTalk Server runtime. The browse test is effectively passing an empty message (which is obviously not a valid message).
  6. To test the HTTP receive fully, use a simple test harness to HTTP POST a valid XML message to the receive location. The message must be a valid XML instance of an installed BizTalk Server Schema and an orchestration should be running that can accept the message (otherwise, an application event log will be generated reporting a "subscription error"). See the procedure in the following section for instructions for using a simple ASP POST test harness.
  7. If the message is consumed, but no IIS error or event log errors are present, then confirm that the required orchestrations are started.

Testing the HTTP POST Sample

A simple ASP-based HTTP POST test application is included in the samples located in the "XML HTTP Test Stub and Test Harness" folder.

Using the Simple ASP POST Test Sample, do the following:

  1. Copy the XMLPostASP folder to C:\InetPub\wwwroot.
  2. In IIS Manager, create a new virtual directory called HTTPPost that points to C:\InetPub\wwwroot\XMLPostASP, and set it to allow execute permissions and directory browsing.
  3. In Notepad, modify open PostXMLdata.asp page.
    Locate the line that sets the requestLocation variable and set the value to the address of the HTTP Receive location, for example:
  4. Save the updated ASP file.
  5. In Internet Explorer navigate to http://machineNameOrIP/HTTPPost /PostXMLform.htm.
  6. Paste into the form the XML message that you want to submit to BizTalk Server and click Submit.
  7. The response form will be displayed onscreen along with the HTTP status value.

A richer ASP.NET-based application for testing HTTP Post and Receive functionality can be found in the samples under the "ASP.NET test stub and harness" folder.

Setting Up HTTP and SOAP Receive

If it is required to run both HTTP and SOAP receive functions on the same Web server, additional configuration is required to avoid the following error:

"The Messaging Engine failed to register an adapter "SOAP". Details: "Registering multiple adapter types within the same process is not a supported scenario. For e.g. HTTP and SOAP receive adapters cannot co-exist in the same process"

To avoid this error, create two separate application pools (one for HTTP adapter and one for SOAP adapter).

ms942199.note(en-US,BTS.10).gifNote
Both application pools may use the same BizTalk Server 2004 Isolated Host User identity.

ms942199.note(en-US,BTS.10).gifNote
The actual host configuration used should be dictated by security and isolation requirements of the production environment.

Setting Up Published Web Service Permissions

When using ASP.NET applications created with the Web Services Publishing Wizard on the Windows Server 2003 platform, errors relating to accessing DLLs during Web services invocation may occur. These errors are typically related to issues with default Windows Server 2003 security, which is tighter than the default security under Windows 2000. For more information about these errors, see the Microsoft Help and Support article called "You Receive a "System.IO.FileNotFoundException" Error When the Client Application Calls a Web Service" on the Help and Support Web site at http://go.microsoft.com/fwlink/?LinkId=43659.

BizTalk Server requires that the process running the Web service is granted the appropriate permissions. Under Windows Server 2003 the default for this is likely to be Isolated Host User, so adding the permissions to the Isolated Host Users Group should resolve this issue.

To add the permissions to the Isolated Host Users Group:

  1. In Microsoft Windows Explorer, locate the %windir%\temp directory.
  2. Right-click %windir%\temp, and then click Properties.
  3. In the Properties dialog box, click the Security tab.
  4. Click Add, select the Isolated Host Users Group, and then click OK.
Runtime Exception on Invoke When Calling into a BizTalk Server 2004 Generated Web Services Interface Errors

If an exception is generated by a call to an ASP.NET application that was created with the Web Services Publishing Wizard, but there are no messages in HAT or the Event Log, then it is possible that the IIS_WPG does not have sufficient permissions to access the ASP.NET files. This typically occurs if the published ASP.NET Web service is not located under the default INETPUB\WWWROOT. The IIS_WPG group needs the same privileges as it has to WWWROOT.

Windows SharePoint Services and BizTalk Server

By default, Microsoft Windows SharePoint® Services installs on the default Web server (typically configured on port 80) and assumes that the entire Web server will be devoted to using Windows SharePoint Services. If (as is commonly the case) you are also hosting Web applications or Web services on the same Web server this will cause issues because Windows SharePoint Services secures the entire Web site (and blocks the Web services and applications).

There are two possible ways to avoid this:

  • Install Windows SharePoint Services on an alternative Web server instance. If the developer environment is running on Windows Server 2003, then prior to installing BizTalk Server, use the IIS MMC to create a new Web server instance (not virtual directory) and assign it to run on an unused port number (like 8000). Then install Windows SharePoint Services to http://<machinename>:8000.
    This operation will keep all the Windows SharePoint Services security on a completely separate Web server from Web services and Web applications.
  • Remove the Web server root from Windows SharePoint Services control. Exclude the <root> of the IIS default Web site as a managed path using the SharePoint Central Administration site by following these steps:
    1. Open the SharePoint Central Administration site (Control Panel | Administrative Tools)
    2. From the home page of Central Administration, under Virtual Server Configuration, click Configure virtual server settings.
    3. Click Default Web Site.
    4. Under Virtual Server Management, click Define managed paths.
    5. To remove the root path as a managed path, click Remove selected paths under Included Paths, and then select the check box next to Root.

BizTalk Server 2004 and Windows XP Service Pack 2

BizTalk Server 2004 will stop functioning after the installation of Windows XP SP2 due to changes in the DCOM security model. You can re-enable BizTalk Server 2004 by modifying the registry. The instructions to resolve this issue can be found on the Microsoft Help and Support Web site at http://go.microsoft.com/fwlink/?LinkId=43660.

BizTalk Server 2004 and .NET Framework 1.1 Service Pack 1

The BizTalk Server 2004 Business Rules Engine assembly will stop functioning after the installation of the .NET Framework 1.1 SP1. To re-enable the Business Rules Engine and resolve this issue, download the hotfix for the Business Rules Engine assembly from the Microsoft Help and Support Web site at http://go.microsoft.com/fwlink/?LinkId=43661.

Enable Runtime Schema Validation

By default BizTalk Server 2004 does not perform schema validation against incoming XML messages entering the BizTalk Server 2004 orchestration runtime. In this configuration BizTalk Server 2004 only examines the root node and target namespace of the incoming document. Comparing every incoming XML instance against its schema is a resource-intensive process and this default setting allows messages to enter BizTalk Server 2004 with the maximum throughput. Typically the default "no validation" setting is used for production environments.

During the development period it is recommended that runtime schema validation is switched on. By validating messages during development, potential errors relating to badly formed messages are caught early in the development process rather than later, when they are typically more difficult to fix.

In some cases it may be applicable to switch on message validation in the production system (especially if messages may be coming from systems that may not have been through a rigorous integration testing process). If runtime validation is switched on, then be aware of the impact on throughput and capacity. Consider using the BizTalk Server performance counters to understand the impact.

To turn on runtime schema validation:

  1. Make a backup copy of the existing BTSNTSvc.exe.config file (found under C:\WINDOWS\Microsoft.NET\Framework\<FrameworkVersion>\config).
  2. Modify the BTSNTSvc.exe.config file by adding the configuration section shown below.
    <?xml version="1.0" ?>
    <configuration>
     <xlangs>
      <Configuration>
        <Debugging ValidateSchemas="true"/>
      </Configuration>
     </xlangs>
    </configuration>
    
  3. Restart the BizTalk Server 2004 service to pick up the changes.

For more details on the runtime configuration, see "Runtime Validation" in BizTalk Server 2004 Help.

Switching On Runtime Schema Validation in Non-Orchestration Solutions

The preceding approach switches on validation of messages entering orchestrations. To enable validation of incoming messages in a "messaging only" solution requires turning on validation by setting the configuration options in the XML and FF disassemblers and by using the XML validator component in pipelines. See "XML Disassembler Pipeline Component" and "Flat File Disassembler Pipeline Component" in the product documentation.

Developing BizTalk Server 2004 Applications Using Remote Desktop

It is possible to develop and debug BizTalk Server applications while running the BizTalk Server developer environment on a remote computer accessed by Remote Desktop or Terminal Services. However, teams using this approach should be aware of the following limitations:

  • Attaching to BTSNTCVS.EXE (for example, for debugging custom functoids or adapters) may cause issues due to the single instance of BTSNTCVS.EXE.
  • Starting and stopping hosts will cause concurrent users issues unless separate hosts are configured for separate users.
  • Visual SourceSafe working folder settings may not function as expected.

If the BizTalk Server environment is running on Windows Server 2003 it is possible to connect to the console session rather than starting a new user session. This can be useful in scenarios where debugging information or the like is being written to the console.

To connect to the Windows Server 2003 console using Remote Desktop, use the "/ console" command-line option, for example:

mstsc.exe <RDP connection file name> /console

Using Virtual PC or Virtual Server to Host the BizTalk Server 2004 Developer Environment

Setting up a BizTalk Server 2004 development environment typically requires the following:

  • Installation of prerequisites
  • Application install process (SQL Server, Visual Studio .NET, and BizTalk Server 2004)
  • BizTalk Server group configuration
  • Installation of tools and utilities

An alternative to performing these steps on every developer's workstation is to use the services provided by Microsoft Virtual PC. Microsoft Virtual PC allows users to run multiple PC-based guest operating systems simultaneously on the host operating system. Virtual PC also allows users to run multiple PC-based operating systems simultaneously. For example a user with Virtual PC installed on a Windows XP computer can run Windows Server 2003 within the Virtual PC. The benefit to a BizTalk Server 2004 development team lies in the fact that Virtual PC uses Virtual Hard Disk files (VHD files) that can be copied between computers. In essence a BizTalk Server 2004 development team can invest time in creating a standard BizTalk Server 2004 developer virtual hard disk and then push that image to every member of the development team who gains a preconfigured BizTalk Server 2004 development environment.

ms942199.note(en-US,BTS.10).gifNote
The installation of BizTalk Server 2004 within a Virtual PC is not a supported configuration. In the event of a BizTalk Server 2004 issue arising when running under Virtual PC only "commercially reasonable support" will be provided. For an issue to be progressed as a possible candidate for a BizTalk Server 2004 code fix, the issue must be reproducible in a non-Virtual PC environment. "Commercially reasonable support" excludes the production of BizTalk Server 2004 code fixes to resolve issues.

For more information, see the Microsoft Knowledge Base article called "Installation of BizTalk Server 2004 onto a virtual machine is not a supported configuration" at http://go.microsoft.com/fwlink/?LinkId=43662.

In light of the lack of full support for BizTalk Server 2004 on Virtual PC it is recommended that a development team using Virtual PC have the capability to reproduce any issues upon a native (non-Virtual) environment.

Follow these steps to create a Virtual PC virtual hard disk containing a BizTalk Server 2004 developer environment:

  1. Ensure that the developer workstations are powerful enough to run the guest Windows Server 2003 operating system. Typically this will require a fast Windows XP workstation with a minimum of 1 GB of physical RAM (2 GB optimal) and approximately 10 GB of local disk storage available.
  2. Create a new Virtual PC Virtual Hard Disk. By default the Virtual Hard Disk type is set to "Dynamically Expanding". If fixed size is used then ensure that a minimum of 6 GB is selected.
  3. Install Windows Server 2003 and SQL Server 2000 on the Virtual Hard Disk, following the instructions in the BizTalk Server 2004 installation documentation.
  4. Install all recommended security updates.
  5. Using the Virtual PC settings, enable the Virtual PC to access the network (network access is required to contact Visual SourceSafe, shared drives, and so on).
  6. Install and configure BizTalk Server 2004.
  7. Install Visual SourceSafe.
  8. Install any additional tools or utilities that the development team should have access to. Installing common tools on the VHD can save considerable time and effort later. See later in this section for a list of useful tools.
  9. Shut down the virtual PC instance and make a secure read-only copy of the Virtual Hard Disk file. This is now the master VHD for the development team.
  10. Copy the VHD file to developers' workstations as required.

Note that the preceding approach will create Virtual Hard Disk images with the same machine name and SID. In most cases this approach will not hamper development if all development is within the Virtual PC environment. (Introducing the Virtual PC Guest operating system onto the network will generate warnings about non-unique computer names.) If unique computer names are required, they can be created by creating a Virtual Hard Disk with a "SysPrep" copy of Windows Server 2003 and then using a scripted install of SQL Server 2000 and BizTalk Server 2004 to allow the efficient production of uniquely named Virtual PC images.

ms942199.note(en-US,BTS.10).gifNote
SysPrep can be found on the Windows Server 2003 server CD.

When developing on a Virtual PC, a useful approach can be to create a second hard disk (VHD file) that holds the source code for projects. If this approach is used across the whole team with the same drive letter assigned to the drive then this provides a consistent set of paths for the solution structures. A major advantage of this approach is that the separate VHD can be significantly smaller and as such easier to copy between remote locations.

For more information about Virtual PC, see the Microsoft Virtual PC 2004 Web site at http://go.microsoft.com/fwlink/?LinkId=43664.

When the BizTalk Server 2004 team development and test environment is put into place, it is important to make certain that the platform and environment is stable. The following sections detail techniques and approaches that will help deliver a stable environment and stable development process.

Backing Up Development Data

First, ensure that the Visual SourceSafe store is robust. Always ensure that the Visual SourceSafe repository is located on a file server that is under a backup regime or on a resilient disk subsystem. In this way all the entities that are checked into Visual SourceSafe will be retrievable in the event of a workstation failure.

Always use Visual SourceSafe to store all the shared deliverables like install scripts, build scripts, and deployment scripts. This ensures that although they are not necessarily assigned to a specific user they are still under a versioning and backup strategy.

The recommended approach to using Visual SourceSafe is to check in code only when it has passed functional testing. This means that a considerable period of time can pass between check-in operations, during which time the code is not on the backed-up Visual SourceSafe disk location. To mitigate the failure of a developer workstation it is important to ensure that a backup of the developer's workstations takes place.

A simple approach to back up developer workstations can be set up by using the robust copy (robocopy.exe) tool that is distributed with the Windows resource kits. You can find this tool on the Windows 2000 Web site at http://go.microsoft.com/fwlink/?LinkId=44527.

Robust copy will perform an incremental, multifolder copy, providing an efficient way to perform simple backup operations.

To set up a developer workstation backup:

  • Copy robocopy.exe into the Windows\System32 folder of all developers' workstations.
  • Create a command file that contains a line similar to the following:
    robocopy c:\SysInteg2004 \\backupserver\SysteInteg2004\DeveloperWorkstation001\ /s /z
    
  • Create a scheduled task to run the task at regular intervals, making sure that the user account used to run the task has appropriate write permissions on the destination backup server.

For developers who are new to integration development, one of the most noticeable differences is the lack of visual feedback when testing integration code. Traditionally a client application or server assemblies can report status or errors and the Integrated Development Environment (IDE) will provide detailed messages and error context information (such as stack contents and traces). Development under BizTalk Server 2004 requires an alternative approach to debugging the runtime.

This section provides debugging and tracing information that will be of use to those new to BizTalk Server 2004 development.

No Activity Checklist

One of the most common scenarios that occurs when developing with BizTalk Server 2004 is that of initiating a business process (for example, dropping an input file into a folder) and then not finding any corresponding output (for example, no corresponding output file in the output folder).

Due to the flexibility available in BizTalk Server 2004 configuration, there are many possible causes for this type of scenario. The following list is provided as a simple checklist to allow a developer to check some of the commonly occurring causes:

  • Check the system event viewer for error messages. The event log is the location that BizTalk Server 2004 uses to record runtime errors and is the best place to diagnose errors with documents being processed. Remember that if your BizTalk Server 2004 configuration has multiple servers you may need to check the event logs of all the servers.
  • Ensure that the BizTalk Server 2004 service is running.
  • Ensure that the in-process host is shown as running in the BizTalk Server 2004 Administrator.
  • If using a file port or file send port, check that the "BizTalk Server 2004 Application User" has "read, write, and modify" permissions on any file locations in use. If this user does not have permissions to create and delete files the receive functions will fail because they cannot guarantee to successfully read or write files.
  • If using a file receive location, ensure that any input files dropped into a file receive location do not have the READ ONLY attribute set. If this is the case the file receive port will disable itself. Remember to refresh BizTalk Explorer to identify if the status of the receive ports has changed.
  • Check that the orchestration is enlisted and running. A common initial oversight is to fail to enlist or start the process that will handle the incoming messages. Use BizTalk Explorer in Visual Studio .NET to ensure that the process you are testing is running.
  • If the deployed DLL has just changed, it is possible that the BizTalk Server 2004 runtime has not yet picked up the latest version and is using a cached copy of the previous DLL. To force the runtime to pick up the new version, recycle the BizTalk Server 2004 hosts or restart the BizTalk Server 2004 service. See "Restarting BizTalk Server Hosts and Services" for more details.
  • Check Health and Activity Tracking (HAT) to see which phases the message is passing through (see "Health and Activity Tracking" for more details).
  • If using an HTTP receive port, check that it is functioning correctly. Use the IIS MMC to navigate to the virtual directory containing the BTSHTPPReceive.DLL. Right-click the BTSHTPPReceive.DLL file in IIS admin and then click Browse. An HTTP page should be returned and an error should be reported in the event log of the receiving BizTalk Server 2004 server, reporting that the runtime received a message but could not process it. The error message shows that the receive function is working but that BizTalk Server 2004 could not successfully process the message (which is to be expected because it had no actual XML content).
  • If using any test harnesses (like the ASP HTTP POST application), ensure that the harness is posting to the correct address. Check that the test harness is posting to the correct address by pasting the address into a browser and ensuring that the address resolves and a page is returned successfully.
  • Ensure that no orchestrations have debug points set that would cause processing to halt.
  • If binding files have been used to deploy a solution, check the installed binding configuration against the expected configuration in case incorrect binding files have been used.
  • If an IIS configuration change has been made (for example, changing an application pool setting), run IISRESET to ensure that any changes have been picked up.
  • Certain BizTalk Server 2004 configuration changes require the host to restart to take effect. If changes have been made to the BizTalk Server 2004 runtime, try restarting the BizTalk Server 2004 service.
  • Check that the appropriate BizTalk Host is running on the appropriate computer. When multiple servers are configured and processes can be bound to specific hosts it is important to confirm that the correct host is started and running.

Health and Activity Tracking

The Health and Activity Tracking (HAT) tool is a valuable tool for understanding the content and status of messages submitted to the BizTalk Server 2004 runtime. For more information about HAT, see BizTalk Server 2004 Help.

It is often useful to be able to use HAT to view the actual contents of a message. This is not enabled by default (due to the resource overhead incurred when recording message bodies). The following steps can aid a developer looking to capture the content of specific messages submitted to the runtime.

To capture message contents using HAT:
  1. Make sure that message tracking is set up to capture the message contents for a specific message:
  2. Start HAT and then select Configuration|Orchestrations.
  3. Select the <orchestration name>. Under "Select the events and data to be tracked" select the "Inbound message bodies" and "Outbound message bodies" check boxes, and then click Apply.
To see messages passing through a pipeline (for example, for a flat file):
  1. Start HAT and then select Configuration|Pipelines.
  2. Select <pipeline name>. Under "Select the events and data to be tracked" select the "Inbound message bodies" and "Outbound message bodies" check boxes, and then click Apply.
To see messages not associated with an orchestration (for example, messages directly bound to the MessageBox:
ms942199.note(en-US,BTS.10).gifNote
This approach also records all messages passing through the default XML pipeline.

  1. Start HAT and then select Configuration | Pipelines.
  2. Select Microsoft.BizTalk.DefaultPipelines.XMLReceive and select the "Inbound and outbound message bodies" check box under "Select the events and data to be tracked."
  3. Select Microsoft.BizTalk.DefaultPipelines.XMLTransmit and select the "Inbound and outbound message bodies" check box under "Select the events and data to be tracked."

Having performed the above configuration changes, message bodies will now be available to be saved within HAT.

  • To view the message body, locate the message in HAT and right-click Save all tracked messages.

Two files are produced for each message saved. The XML file contains the context properties of the message (internal message ID, send location, and so on). The .OUT file contains the actual message.

Be aware that messages may not be available for up to a minute after sending, because a SQLAgent job runs once a minute to make message bodies available. If the SQLAgent is not running, the message bodies will not become available. Messages that passed through BizTalk Server prior to the configuration change to save message bodies will not be available (because the message body was not recorded).

Business Monitoring with HAT

It is worth noting here that HAT can also be used for business operations monitoring as well as the developer-based tasks described above. It is common to set up saved queries that display key indicators. For example, creating a saved query that lists the number of a specific business transaction within the last hour can act as a simple method of monitoring for unexpected behavior (for example, zero messages in 24 hours may suggest a transport failure in an upstream system).

Working with Suspended Messages

This section addresses the handling of suspended documents that need to be edited and resubmitted. This can often happen when a malformed message is sent into the integration solution. This section provides an outline of how messages could be accessed, modified, and sent back into the system.

Subscribing to Events

BizTalk Server provides scriptable management capability via Windows Management Instrumentation (WMI) classes. These classes are used to programmatically access the administrative functions available in Microsoft BizTalk Server 2004.

BizTalk provides a WMI Event, MSBTS_ServiceInstanceSuspendedEvent. You can set up a listener to take action when a message is suspended. The following code fragment provides an example of how to listen for these messages:

string scope = "root\\MicrosoftBizTalkServer";
string wqlQuery = "Select * from MSBTS_ServiceInstanceSuspendedEvent";
watcher = new ManagementEventWatcher(scope, wqlQuery);
watcher.EventArrived += new EventArrivedEventHandler(onEvent);
watcher.Start();

This code fragment assigns an event handler that can take action when an event is fired. There is also another event, MSBTS_MessageInstanceSuspendedEvent, but this only occurs for a BizTalk Message Queuing (MSMQT) message instance.

Write Suspended Messages to File

The event that is called when suspended messages occur can be used to write out the message to a file using another built-in BizTalk method, MSBTS_MessageInstance.SaveToFile. The following code provides a snippet of saving a message to file. After messages are saved off, they can be consumed by another receive location.

public void onEvent(object sender, EventArrivedEventArgs e)
{
 // Before calling the following code, query and return all the messages instances                         
 // associated with the particular service instance that was suspended.
// Construct full WMI path to the MSBTS_MessageInstance using the message guid
string strInstanceFullPath =              "root\\MicrosoftBizTalkServer:MSBTS_MessageInstance.MessageInstanceID='{" + MessageInstanceId.ToString() + "}'";
// Load the MSBTS_MessageInstance
ManagementObject objSvcInst = new ManagementObject(strInstanceFullPath);
// Invoke "SaveToFile" method to save the message out into the specified folder
objSvcInst.InvokeMethod("SaveToFile", new object[] {strOutputFolder});
}

Developing

When developing BizTalk Server 2004 solution orchestrations for the first time there are several key concepts and techniques it helps to understand. These concepts are sometimes not immediately clear from the product documentation and so are listed below. This is not an exhaustive list but represents some of the common topics that are helpful to a new BizTalk Server 2004 developer.

  • Messages are absolute in BizTalk Server. This means that after a message has been created (or received by BizTalk Server) it cannot be changed or modified. To modify a received message it is required to create a new copy of the message.
  • Why do Message Construct shapes exist? As mentioned, messages are immutable. This raises the obvious question, when creating a new message, at what point does it become immutable? The short answer is that the message is immutable after the Message Construct shape that created it is complete. All new messages must be created in a Message Construct shape that specifies the message instance to be created. When this shape completes, the message becomes immutable.
  • How do I create new messages? It is not possible to simply perform a "create new instance of a message type X" within an orchestration. The reason for this is the complexity of determining how to populate an instance of a message (especially when the message may contain optional or repeating structures). In practice it is possible to create new messages by using the following approaches:
    • A map with destination schema of type X, but no links
    • Call custom .NET code that uses the document object model to create a new message instance
    • In an expression shape, instantiate the XML Document Object Model (DOM) and create a simple message using strings to populate the DOM
    All the above must take place inside of a Message Construct shape.
  • How can I create arrays within an orchestration? Orchestrations cannot handle arrays. If the orchestration needs to create in an array (for example, construct an array of facts/documents for sending to a policy in the Business Rules Engine), define a variable of type System.Collections.ArrayList.
    The following code shows how to create arrays within an orchestration from the Business Rules Framework Programmers Guide:
    Declare an orchestration variable FactsArrayList of type System.Collections.ArrayList and mark it as created using the default constructor.
    FactsArrayList.Add(Doc1)
    FactsArrayList.Add(Doc2)
    Policy = new Microsoft.RuleEngine.Policy("ValidateSalesOrder");
    Policy.Execute(FactsArrayList.ToArray());
    
  • How do I get and use file names within an orchestration? In your orchestration, create an expression and store the message context property value in a variable with code that is similar to the following:
    //CreditCardApplicationInfoMsg is the name of your instantiated incoming message. fname is just a variable that you declare.
    fname = CreditCardApplicationInfoMsg(FILE.ReceivedFileName);
    
    To modify the outgoing file name, use the %SourceFileName% macro in the send port and modify the FILE.ReceivedFileName property in an expression shape.
  • How do I use the reserved word "message"? "Message" is a reserved word. Often the name "message" is in schemas and needs to be referenced in an expression. Since "message" is a reserved keyword for orchestrations, it is necessary to delimit (prefix) it with the "@" character, as follows:
    MyXML.@message != ""
    
    This approach is valid for any reserved word that needs to be referenced in a message.
  • Can messages be split? Messages can be split by using an envelope. There is a sample in the SDK, under <Samples Path>\Pipelines\AssemblerDisassembler\EnvelopeProcessing\. Alternatively, you can search on "EnvelopeProcessing (BizTalk Server Sample)" in BizTalk Server 2004 Help.
  • What is valid in the orchestration expression shape? The following list provides rules for valid orchestration expression shapes:
    • "if" and "while" statements are handled by the decision shape. You cannot put an "if" or "while" into an expression shape.
    • Comments work fine, but you need at least one statement in the expression box.
    • Simple types (integer, string, floating point) cannot have the dot operator applied to them (that is, there is no member access).
    • For non-simple types, you can only access public member functions and properties and static literal fields.
    • Compound assignments (+=, -=, *=, etc) are not supported.
    • More than one assignment operator in a statement is not supported.
    • Assignment within an "if" or "while" predicate is not supported.
    • Increment and decrement are not supported (++, --).
    • For message parts, the only member access allowed is on distinguished fields.
    • Indexers or parameterized properties are not supported.
    • Delegates and events are not supported.
    • For each, for, do/while, break and continue are not supported.
  • Why is it important to serialize .NET assemblies? BizTalk Server 2004 orchestrations can be dehydrated to SQL Server during their execution when they meet certain conditions (such as waiting for a message and Delay shapes). This allows the BizTalk runtime to flush inactive orchestrations out of memory, thus improving scalability and performance.
    It is recommended that any .NET assemblies being called from BizTalk Server be marked as Serializable to facilitate this wherever possible. If it is not possible (for example, due to the assembly being from a third party), you must invoke these assemblies from within an Atomic Transactional scope.
    If you can mark your assembly as Serializable then you do not need to follow these steps:
    1. Right-click your orchestration surface and click Properties.
    2. Change the Transaction type to "Long Running."
    3. Drag a scope onto the orchestration and configure it to be a transaction type of Atomic. This tells BizTalk Server that any operations within this scope cannot be interrupted.
    4. Place the usage of the non-serializable .NET assemblies within this scope.
  • How can I get and set file names within an orchestration? You can also use the FILE.SourceFileName property for reading and redefining the file name to be used on output. Use the macro %SourceFileName" in the file transport.

Outputting Debug and Trace Information

When developing orchestrations it is important to be able to understand the status of an orchestration both during and after execution. HAT provides the capability to view the flow of an orchestration after it has completed and to place a breakpoint within an orchestration (see "Health and Activity Tracking" under "Operations" in the product documentation for more details). Although HAT is effective, it can also be useful to develop orchestrations that write out debugging information as they run.

One of the techniques most commonly used when developing debugging traditional code is to write out useful data during code execution to allow observation of the data and processes that are actually executing. The same functionality can be achieved in BizTalk Server 2004 orchestrations by using the following technique.

Within the .NET class library, the System.Diagnostics.Trace class allows the outputting of trace and debug information to listeners, which can capture and log the information. It is possible to write listeners using .NET code (see "Debug Class" in the .NET documentation), but it is also possible to use readily available utilities.

One such utility is DebugView, which is a freeware utility available from SysInternals at http://go.microsoft.com/fwlink/?LinkId=43665.

When DebugView is running, it will catch and log the output of any debug statements, allowing the developer to view progress and actual data within the process. Note that debug statements also exist within other Microsoft applications (including BizTalk Server 2004), so expect to see debug output from other applications too.

The following steps describe how to output a hard-coded string containing a process name and version number:

  1. In Visual Studio .NET, open an existing orchestration that can successfully be deployed and executed.
  2. Create a new expression shape below the start shape of the orchestration.
  3. Edit the expression as follows:
    System.Diagnostics.Trace.WriteLine("Entering Orchestration)");
    
  4. Compile, deploy, and start the orchestration as normal.
  5. Start DebugView or an alternative debug listener.
  6. Activate the BizTalk Server process and observe the output in DebugView as the orchestration is started by the BizTalk Server runtime.

Output Assembly Version to Aid Tracing, Debugging, or Exception Handling

It can be useful to have orchestration output information about the assembly that is currently running. The following example code displays information about the current assembly. You can use the example code during development to provide a generic approach to listing assembly information. This sample writes to the event log rather than to a trace or debug listener.

  1. Within an orchestration add the following variables:
    var_Assembly of type System.Reflection.Assembly
    var_AssemblyName of type System.Reflection.AssemblyName
    var_version of type string.
    
  2. Within the same orchestration add the following to an expression:
    var_Assembly = System.Reflection.Assembly.GetExecutingAssembly();
    var_AssemblyName = var_Assembly.GetName();
    var_Version = var_AssemblyName.Version.ToString();
    
    System.Diagnostics.EventLog.WriteEntry("BTS Sample", "Version is: " + var_Version);
    
    //Must set .net types to null to prevent attempted serialization if the orchestration dehydrates.
    
    var_Assembly = null;
    var_AssemblyName = null;
    
    

Debugging Message Content from an Orchestration

The trace.writeline technique described above can also be used to write out the contents of messages and variables at run time. This can be extremely useful when working with complex orchestrations, for example, multi-map transformations where it would normally not be possible to observe the intermediate messages produced by maps inside the orchestration.

The following procedure describes how to write out the contents of a populated orchestration message (named myMsg) to a debug listener:

  1. Within an orchestration add the following variables:
    myString declared as an orchestration variable of type 'string'
    xmlDoc declared as an orchestration variable of type 'xmlDocument' (type found under .Net classes, System.XML)
    myMsg is the orchestration message whose contents are to be output
    
  2. Convert the message into a string and output the string in an expression block by adding the following expression to an expression shape:
    xmlDoc = myMsg; //create XML doc from the BTS message
    myString = xmlDoc.OuterXml; //get a string representation of the XML
    System.Diagnostics.Trace.WriteLine(myString); // output the string
    
  3. Run the orchestration with a debug listener running. The XML representation of the message will be output.

It is also worth noting that it is possible to perform the reverse of the "Message to string" operation and create a BizTalk Server 2004 message with an XML string.

To convert the string back to a document in a message construct, use the following code:

xmlDoc.LoadXml(myString);
myMsgFromString = xmlDoc;

To achieve this, the following requirements are necessary:

  • The orchestration message myMsgFromString must be defined as having a type derived from a valid schema.
  • The XML being assigned to the message should conform to the schema of the message.
  • The expression shape needs to be enclosed within a message assignment shape, which is explicitly creating message myMsgFromString.

Debugging Maps

Viewing the map output is an important step when testing maps. The resulting XML produced by the map is saved to disk, even if the output is not valid according to the destination schema. Examining this output can be valuable in understanding why the output of the map is not valid.

Compare the resulting output XML against the schema definition to determine why the error is occurring. The map output document can be seen by looking in the temporary location for the output of a map, for example:

C:\Documents and Settings\<username>\Local Settings\Temp\_MapData

The output document is also available by CTRL-clicking the file name in the output window after a performing the test map operation.

Using Map XSLT to Aid Map Debugging

When developing maps, if map links or functoids do not seem to be producing the expected output or are producing errors, then consider examining the XSLT to work out what node data is being processed and where the results are being written. The XSLT is produced when validating a map and it represents the actual mapping that the BizTalk Server 2004 runtime will perform. By reading the generated XSLT it is sometimes possible to observe the reason that a map is producing unexpected output.

The XSLT is usually quite simple to follow, typically consisting of many simple XPATH queries. By reading the XPATH statements it is often possible to clearly see the input values and any operators followed by the nodes in which the output is written.

The XSLT is produced during a map validation and is located at

C:\Documents and Settings\<username>\Local Settings\Temp\_MapData

Debugging Custom .NET Methods Within the Scripting Functoid on a Map

BizTalk Server 2004 supports the creation of maps that use the scripting functoid. Users can create their own .NET methods and use them from within the scripting functoid to allow better reuse of custom code and functionality. Debugging these functions when testing a map can be very useful to determine if your custom map functions are working properly, but the steps to debug these custom assemblies are not obvious. Here are the steps you need to take to directly debug your scripting assembly while testing your map:

  1. Open your assembly class in its own Visual Studio .NET development environment. Build the assembly normally, making sure that the build is in debug mode and a symbols file is created.
  2. Deploy your custom assembly to the global assembly cache normally. Make sure you deploy the version in bin\debug.
  3. Open up your BizTalk project in a second Visual Studio .NET development environment, reference your custom assembly DLL in the same directory above, and configure your scripting functoid as normal (you should see it as a choice in the list of available assemblies).
  4. Return to your custom assembly environment, and under processes, attach to your BizTalk DEVENV process. Initially, when you set a breakpoint, the environment will say "symbols not loaded". This is normal, because the assembly is not loaded until it is actually invoked by the mapper.
  5. Return to your BizTalk project and test the map (using right-click, Test Map). If you watch your assembly environment the symbols get loaded and the "?" goes away, at which time your breakpoint should hit.
  6. Debug in the normal fashion. Your breakpoint will be hit every time the functoid is called.
Error: "Map contains a reference to a schema node that is not valid"

This design-time error can occur when the map being opened uses a project dependency for one of its schema. If this error occurs ensure that the dependency has been recompiled to reflect any recent changes.

To understand why this error arises, consider the following scenario:

  • Project Y references project X. Project Y uses schema X which are contained in Project X.
  • The schema information used by the map editor in Project Y is actually contained within the compiled DLL produced by project X.
  • This means that if the DLL from project X is deleted then the maps in project Y will not compile until project Y's DLL is recompiled.

This is significant when Visual SourceSafe is being used to store projects. Visual SourceSafe does not store DLLs by design, and when retrieving a BizTalk project containing schema the project will need to be recompiled to produce the DLLs that dependent projects need.

Error: "Element cannot contain text or white spaces. Content model is empty"

This runtime error can occur when using Complex Content elements (like <ANY> or <SEQUENCE>). Complex content elements are not allowed to contain text unless their 'Mixed' property is set to 'True'.

For example, if the schema uses complex content elements, the XML <Root> </Root> (note the space between the nodes) will produce the above error. The XML <Root/> will not because no white space exists. To avoid this error set the Mixed property of the root node of the schema to true, as shown in the following figure.

Figure 8 Schema root node mixed property

n/a

Debugging Adapters and Pipelines

You can debug adapters by using a similar approach to that outlined in the preceding section. Be aware that although adapters do not need to be in the global assembly cache for the runtime to find them, they do need to be in the global assembly cache for Visual Studio .NET to find them when debugging. Make sure that adapter assemblies are in the global assembly cache before attaching to the BTSNTSvc.exe to debug them.

Debugging Web Services Called by BizTalk Server

A common integration requirement is to a call a Web service from the orchestration, and then debug the Web service to determine the functionality taking place within the Web service when called from BizTalk Server 2004. The following steps describe how to debug this scenario. The orchestration is shown in the following figure. It receives a message from the MessageBox, transforms it into the message expected by the Web service, sends it to the Web service, and receives a response.

Figure 9 Orchestration calling a Web service

n/a

To debug the Web service, perform the following steps:

  1. Open Visual Studio.
  2. Open the solution (or project) that contains your Web service.
  3. Set the breakpoint (in our example we set it at line 86).
  4. On the Debug menu, click Processes. The Processes dialog box appears.
    Figure 10 Process dialog box
    n/a

  5. Select aspnet_wp.exe (The ASP.NET worker process represents the ASP.NET runtime environment. It consists of a Win32® unmanaged executable named aspnet_wp.exe, which hosts the .NET common language runtime (CLR). This process is the executable you need to attach to in order to debug ASP.NET applications.)
  6. Select Attach. The Attach to Process dialog box appears.
    Figure 11 Attach to process dialog box
    n/a

  7. Select Common Language Runtime and then click OK.
    The Attach to Process dialog box closes.
  8. In the Processes dialog box, click Close.

The Web service is now ready to be debugged. Send the message to the orchestration and Developer Studio will stop at the breakpoint.

Debugging Pipelines

Pipeline assemblies are often used in BizTalk Server 2004 solutions to provide special handling of files before they are sent into or out of the BizTalk MessageBox. Debugging pipeline assemblies in Visual Studio .NET is critical in order to quickly troubleshoot and fix problems.

Copying and Debugging Assemblies

After you build the pipeline assembly successfully, you will need to copy the files to the "Pipeline Assemblies" directory in the BizTalk Server installation directory (C:\Program Files\Microsoft BizTalk Server 2004\Pipeline Assemblies). If you are updating an assembly that has already been installed and you cannot copy over the existing assembly there are two possible causes:

  • BizTalk process may have the assembly in use. First stop and start the BizTalk service before copying over the existing assembly.
  • A BizTalk project with a pipeline that uses your assembly may have it in use. The Visual Studio .NET IDE may be using the design-time aspects of the pipeline assembly. To release it, close Visual Studio .NET. (It may be necessary to close all instances, not just the pipeline project.)

After the file has been copied, attach to the BizTalk process as follows:

  1. With the pipeline assembly project open, on the Tools menu, click Debug Processes.
  2. From the Available Processes list, select the BizTalk Server process (BTSSvc.exe) and click Attach.
  3. In the Attach to Process dialog box, in the "Choose the program types that you want to debug" list, select Common Language Runtime is selected.
  4. Click OK.
  5. Click Close.
ms942199.note(en-US,BTS.10).gifNote
After a changing a pipeline, the BizTalk Server 2004 runtime does not pick up the change until it refreshes its configuration. To test the changes immediately, recycle the appropriate hosts (using recyclehosts.vbs as described in Table 20 under "Redeployment Scripts").

ms942199.note(en-US,BTS.10).gifNote
If the "redeploy" option is set to true this allows a modified pipeline to be redeployed without undeploying a previous version. However be aware that this redeploy action resets any send or receive ports that used this pipeline to "PassThrough". After the redeploy it is necessary to reassign the port to the custom pipeline.

Helper Classes

It can be useful to include helper classes within a BizTalk project to perform business logic tasks or functions that aid the development process. A typical task of a developer helper class would be to write messages out to the file system to aid debugging and diagnostics.

Included in samples is a template of a general BizTalk Server helper class that manipulates messages. It provides limited functionality to aid the debugging of messages, including a "DumpMessageToFile" function that writes a BizTalk Server message to the file system.

This sample can be modified to handle more complex functionality. To use the sample within a BizTalk project complete the following steps:

  1. Compile the project.
  2. Copy GeneralHelper.dll to a permanent location on the local file system.
  3. Add the DLL to the global assembly cache so it is visible to the BizTalk Server runtime using:
    GACUTIL /I GeneralHelper.dll
    
  4. Add a reference to the GeneralHelper.dll in the orchestration project.
  5. Within an expression shape in an orchestration use the following syntax:
    DebugHelper.Utils.DumpMessageToFile(<MessageName>,"<path>","<filename>",<timestamp>);
    
    <MessageName>: String, Name of the BizTalk Server message in the orchestration to be written out
    <path>: String, Path for file to be written to, with backslashes denoted with "\\"
    <filename>: String, Filename for the resulting file
    <timestamp>: Boolean, prepends a timestamp to the filename to create quasi-unique filenaming for messages
    
    Another Example:
    DebugHelper.Utils.DumpMessageToFile(AccountUpdateIncoming,"c:\\tmp"," AccountUpdateIncoming.xml",true);
    

Useful Development Tools

The following tools are useful to have available within the BizTalk Server 2004 developer's environment:

  • BizTalk 2004 Management Tool. This stand-alone tool provides functionality similar to the BizTalk Explorer capabilities in Visual Studio .NET, and as such is an extremely useful tool when configuring or debugging a server environment that does not have Visual Studio .NET available. The tool enables the user to manage assemblies, orchestrations, ports, deployment, and messaging. You can download the BizTalk 2004 Management Tool from the GotDotNet Workspaces Web site at http://go.microsoft.com/fwlink/?LinkId=44998.
  • BizTalk Server 2004 Configuration Documenter. This tool runs against the Configuration database of a BizTalk Server 2004 group and documents the configuration of all the deployed BizTalk Server 2004 solutions within the group. This tool can be extremely useful in documenting and validating deployments, and it also provides a basis for solution documentation. You can download BizTalk Server 2004 Configuration Documenter from the GotDotNet Workspaces Web site at http://go.microsoft.com/fwlink/?LinkId=44999.
  • DebugView. This tool is described in the debugging sections. You can find the DebugView tool on the SysInternals Web site at http://go.microsoft.com/fwlink/?LinkId=43665.
  • TCPTrace. TCPTrace provides the ability to view and log the data flowing between TCP ports. When using BizTalk Server 2004 and the HTTP or SOAP protocols, it can be extremely useful to view the actual data flowing between systems and the integration layer. By configuring TCP to act as a proxy between the integration layer and an application it is possible to capture HTTP and SOAP messages and then examine the schema and content of these messages.
    You can download the TCPTrace tool on the Pocket SOAP Web site at http://go.microsoft.com/fwlink/?LinkId=43666.
  • Windows Explorer BizTalk Server Extension. The Windows Explorer BizTalk Server Extension tool is part of the BizTalk Server 2004 installation process, but by default is not registered. To register the tool, close all instances of Internet Explorer and run the following command from the command prompt:
    regsvr32 "c:\Program Files\Microsoft BizTalk Server 2004\Developer Tools\BtsAsmExt.dll 
    
    This tool adds a BizTalk Server search pane to the standard Windows Explorer. You can access the new pane from the folders bar or by using the Windows Explorer View menu, pointing to the Explorer bar, and clicking BizTalk Server Search.
    This tool also allows you to search across the deployed assemblies for any of the BizTalk artifacts and to view detailed information about the artifacts found. The following figure shows the location of the tool.
    Figure 12 BizTalk Server Extension tool
    n/a

  • BizTalk Server Documentation Tools. See "Documenting BizTalk Server 2004 Systems" for more information about these tools.
  • ASP HTTP Test Stub and Test Harness. Two simple ASP pages are included in the samples with this document and include the following resources:
    • A test harness to POST an XML message to an HTTP URL and display the response.
    • A test stub to provide an XML response (taken from an XML file) to an HTTP GET request.
    These simple ASP files are designed to provide easy-to-modify templates to create test stubs and harnesses. You can find the ASP files in the samples folder called "XML HTTP Test Stub and Test Harness."
  • ASP.NET HTTP Test Stub and Test Harness. You can find an ASP.NET base test stub and harness in the samples folder. This test tool allows you to simulate the source and target application to perform end-to-end testing. The source application initiates the business process by posting data to the receive location of the BizTalk Server and waits for the response from the BizTalk Server. The business process then submits the data to the target application by using the "Solicit Response" port and listens for the response from the target application.
    This ASP.NET application provides the same functionality as the ASP version, but could be significantly extended by using further .NET functionality to meet project requirements. The following figure shows operations that you can perform with the stub and harness.
    Figure 13 ASP.NET stub and harness operations
    ms942199.Local_-716283091_figure13(en-US,BTS.10).gif

    You can find the ASP.NET HTTP test stub and test harness tools in the samples folder under "ASP.NET HTTP test stub and test harness."
    Additional BizTalk Server 2004 tools are currently being collected on the Web under the collective name of "BizTalk Server 2004 Power Toys," and a search for these may reveal more useful tools. BizTalk Server 2004 tools were also written for the Microsoft "BizTalk Tools Competition" in September 2004, and these can also be found by searching the Web.

This section provides information, toolkits, templates, and tools useful to teams looking to develop efficient build and deployment phases.

Automating Developer Deployment Tasks

When a BizTalk Server 2004 developer is developing a solution, there is a common requirement to perform the following steps: build, deploy, test, and undeploy. The Visual Studio .NET BizTalk Explorer assists in carrying out these tasks, but does not provide an approach to automate these often-repeated tasks. This section provides some techniques to improve the efficiency of these tasks.

Redeployment Scripts

BizTalk Server 2004 provides command-line and WMI interfaces to the BizTalk Server 2004 deployment functionality. This allows the creation of command and script files. These files can enable a developer to redeploy simple or complex solutions with a single operation.

These scripts are also beneficial when working with orchestrations that have dependencies upon other orchestrations. BizTalk Server 2004 enforces the dependencies when starting/stopping and deploying/undeploying orchestrations, and a script is an efficient way to ensure that the dependency order is always followed.

To aid the creation of simple redeployment scripts, the samples folder "Simple Redeployment Scripts" contains scripts that were created based upon the SDK samples or the WMI documentation. Where files were modified from the SDK samples, this was done to provide a more reusable or flexible version than the SDK original.

The following table lists the simple build and redeployment template scripts.

Table 20 Simple build and redeployment template scripts

Type Description

deploy.cmd

Sample command file to perform the following steps:

  • Add helper assemblies to the global assembly cache
  • Add BizTalk Server assemblies to the global assembly cache
  • Deploy BizTalk Server assemblies
  • Import BizTalk Server bindings
  • Start BizTalk Server orchestrations
  • Start ports not associated with an orchestration
  • Restart BizTalk Server service

undeploy.cmd

Sample command file to perform the following steps:

  • Stop ports not associated with an orchestration
  • Stop BizTalk Server orchestrations
  • Undeploy BizTalk Server assemblies
  • Remove ports not associated with an orchestration
  • Remove BizTalk Server assemblies from the global assembly cache
  • Remove helper assemblies from the global assembly cache

ControlOrch.vbs

A modified version of the VBScript files to start and stop orchestrations as supplied with the BizTalk Server SDK (found under C:\Program Files\Microsoft BizTalk Server 2004\SDK\Samples\Admin\WMI).

Usage:

ControlOrch.vbs OrchestrationName AssemblyName <start/stop>

ControlRecPort.vbs

A modified version of the VBScript files to start and stop receive ports supplied with the BizTalk Server SDK.

ControlSendPort.vbs

A modified version of the VBScript files to start and stop send ports supplied with the BizTalk Server SDK.

RemoveReceivePort.vbs

An unmodified version of the VBScript files to remove receive ports supplied with the BizTalk Server SDK.

RemoveSendPort.vbs

An unmodified version of the VBScript files to remove send ports supplied with the BizTalk Server SDK.

RecycleHosts.vbs

Script to recycle the in-process application hosts. This is a faster way to get the BizTalk Server runtime to pick up a new version than stopping and starting the BizTalk Server service.

TerminateSuspended.vbs

Terminates all suspended orchestrations. A BizTalk Server 2004 assembly cannot be undeployed if there are any instances of orchestrations (suspended, running, or terminated) that reference the assembly. This script is a quicker (but more dangerous) way of terminating instances than using HAT. Use with caution. Note: Never use in a production environment.

Terminate*.vbs

As above but terminates orchestrations with various differing status (including TerminateAll.vbs). These scripts are quicker than using HAT but should be used with care. Note: Never use in a production environment.

To aid the reusability of the above scripts a team may wish to parameterize the deploy and undeployment scripts to accept paths for the DLL and binding file folder locations. In this way the same scripts can be reused from a deployment package that contains the final compiled DLLs and production binding files in alternative locations.

Notice that the order of the operations is reversed with the undeployment script. This extends to the order for stopping orchestrations and order of assembly uninstallation. This ensures that dependent orchestrations and assemblies are handled in the correct order to avoid undeployment errors.

The deploy.cmd and undeploy.cmd scripts rely on the GACUTIL.exe tool being available to install assemblies. The easiest way to achieve this on a developer workstation is to run the scripts with the "Visual Studio .NET 2003 Command Prompt," which has access to the .NET command-line tools. Alternatively GACUTIL.exe can be included in the scripts folder.

The sample scripts are provided as samples to demonstrate the steps required. You can optimize the scripts for performance on a local developer workstation by using a single BTSDEPLOY operation to deploy, add to the global assembly cache, and bind in one step and by removing the operations to delete ports.

BizTalk Project "Redeploy" Option

A BizTalk project can be configured to allow the redeployment of an updated version of the project on top of a previously deployed version. This option can be found under "Project properties" in the "Configuration" section. However, the redeploy option still insists that the previous version of the orchestration is unenlisted. When developing a solution that requires the redeployment and testing of a single orchestration a simple process can be developed to allow quick deployment of a new version. This approach requires the VBScript scripts described in Table 20 under "Redeployment Scripts."

Ensure that the BizTalk Server Projects redeploy option is set to true. When a new version of the project needs to be deployed, run the following code from the command prompt:

Cscript ControlOrch.vbs orchname assmbname stop
<Deploy the solution (this will perform a build if required)>
Cscript ControlOrch.vbs orchname assmbname stop
Cscript RecycleHosts.vbs

By using the preceding approach you can redeploy a simple project and force the BizTalk Server runtime to use the new version.

Quick Redeploy of Local Developer Workstation

When developing BizTalk Server 2004 solutions, there is no way to test a modified orchestration without actually deploying and binding the resulting assembly. This means that changing just one line of code requires you to stop, unbind, rebuild, redeploy, rebind, and restart an assembly. The redeployment scripts listed above provide an automated "one-step" operation to achieve the necessary functionality; however, you can use an alternative approach in some circumstances.

ms942199.note(en-US,BTS.10).gifNote
The above approach is not officially supported, and should be restricted to local development workstations.

Assumptions and constraints:

  • This approach assumes that an orchestration has been developed and has already been deployed to the global assembly cache and Configuration database, and the orchestrations have been bound.
  • This approach only works when the changes to an orchestration do not affect the preconfigured binding information. For example, the modification cannot include adding a new port that would require additional binding information.

Steps to set up quick redeploy:

  • Using Control Panel | System | Advanced | Environment Variable, set the system environment variable DEVPATH to the project binaries folder. For example:
    DEVPATH="C:\SysInteg2004\AccountUpdate\bin\development\"
    
    ms942199.note(en-US,BTS.10).gifNote
    The path must include the trailing backslash.

  • Edit the machine.config file (found under C:\WINDOWS\Microsoft.NET\Framework\<FrameworkVersion>\config) and set <developmentMode developerInstallation="true"/> under /configuration/runtime.
    ms942199.note(en-US,BTS.10).gifNote
    You may need to add the <runtime> and <developmentMode> elements to the machine.config file.

  • Restart the developer workstation to ensure that all changes take effect.

From this point on the assembly is loaded from the DEVPATH location, and not from the global assembly cache. To make a simple change to the assembly, simply recompile the updated solution and restart the BizTalk Server 2004 service or the BizTalk Host that is hosting the orchestration.

For more information about the DEVPATH environment variable and machine.config settings, see Locating Assemblies Using DEVPATH in the MSDN Library.

Alternative Quick Redeploy

You can also use the following approach when making internal changes to an orchestration that does not require port binding changes. As in the quick redeploy option above, this approach is purely a development shortcut and is not officially supported. Perform the following steps:

  • Make changes to project
  • Rebuild project
  • Shut down host instance
  • Open assembly explorer (that is, c:\windows\assembly using Windows Explorer)
  • Locate previous version of assembly and delete
  • Drag and drop new version of assembly from project bin\development folder into assembly explorer
  • Restart host instance
  • Test new version

Restarting BizTalk Server Hosts and Services

When developing BizTalk Server on a local workstation, it is common to make a change and then redeploy the updated assembly to retest the functionality. Because the BizTalk Server runtime loads assemblies into memory periodically, BizTalk Server does not load up the new version until it reloads the assemblies, unless the updated version has an updated assembly version. To ensure that the BizTalk Server runtime picks up the updated version immediately, stop and restart the BizTalk Server runtime hosts.

Use the recyclehosts.vbs script found in the "Simple Redeployment Scripts" samples folder that accompanies this document.

Alternatively, you can start and stop the BizTalk Server 2004 runtime by creating a command file called BTSreset.cmd that contains the following two lines of code:

net stop "BizTalk Service BizTalk Group : BizTalkServerApplication"
net start "BizTalk Service BizTalk Group : BizTalkServerApplication"

Recycling hosts is quicker than starting and stopping the BizTalk Server 2004 service and is recommended, unless you have modifications such as BizTalk Server 2004 runtime configuration changes that need to be picked up quickly.

Automating Build and Deployment Version Numbering

As the "Team Development" section of this document discussed, a development team may wish to automate the updating of project version numbers and binding file version numbers. This section provides some tools to assist in these processes.

Patching Binding Files

When a development team is producing frequent build and deployment operations on multiple BizTalk projects with the version information on the DLLs changing frequently, it can be helpful to automate the modification of the version numbers or physical addresses in binding files.

This process can be automated by using a command-line tool to change version information in the binding file. This is typically carried out in an automated fashion by using a .bat or .cmd file before using BTSDEPLOY to deploy.

Using PatchXml

PatchXml is a command-line tool that provides bind file version modification. PatchXml can be found in the samples files that accompany this document under the "Patching Binding Files\PatchXml Binding File Change Tool" folders. The source is provided to allow you to add enhancements to meet project requirements.

To see usage instructions for PatchXml, run it from a command prompt without arguments. PatchXml can replace a single element or attribute in any XML file by using an XPath expression to identify the element or attribute to replace. For example, you can run PatchXml to change version attribute information in a BizTalk binding.xml file as shown in the following example.

Attribute replacement:

patchxml In.xml Out.xml /BindingInfo/ModuleRefCollection/ModuleRef/@Version 2.0.0.0.

Alternatively, replace the transport address in the binding file that may differ between development and test or production environments. The following example shows this alternative.

Element replacement:

patchxml In.xml Out.xml //PrimaryTransport/Address http://localhost/bogus

For more information about XPath expressions, see XPath Examples at MSDN.

Updating Project Version Number and Binding Using Scripts

In this approach both the BizTalk project files and associated binding files version information is updated by using scripts. The development team may wish to provide developers or build managers with a daily build process that automatically versions the BizTalk projects and binding files as a scripted operation. This can typically be carried out by using scripts that use the extensibility of the development environment.

Using BizTalk Server 2004 and Binding Versioning Scripts

The accompanying samples provide two scripts that can be run to modify the build number (that is, assembly version information) for all .btproj files in a file hierarchy and then change the version attributes in the binding files to match. These samples can be found in the samples under "Updating project version number and binding using scripts\BTPROJ and Binding File Versioning Scripts" folders. Remember to update any paths and solution names prior to using the sample.

The following table describes the version-updating scripts.

Table 21 Description of version-updating scripts

Script Function

Localsetversion.vbs

Reads a build number from a file and looks for all .btproj files in a directory tree and sets the version number.

ChangeBindingFile.vbs

Changes various attributes of a binding file to suit builds or different environments (assembly version, send port, host, user group, port map version, etc. elements).

DEVENVbuild.cmd

A sample build file that uses the DEVENV command-line option to compile a solution.

ms942199.note(en-US,BTS.10).gifNote
The build number changes to the *.btproj files will cause Visual SourceSafe issues if you try to open these projects in Visual Studio .NET to compile. To avoid these warnings compile your BizTalk projects by using DEVENV.exe from the command line or a batch file.

For more information about using DEVENV see "Devenv Command Line Switches" in Visual Studio .NET Help.

Manual Build and Deployment Checklist Example

The following section provides a sample checklist for a team to complete the following common BizTalk Server build and deployment tasks:

  1. Build a master BizTalk solution (composed of several BizTalk projects).
  2. Create a deployment package from the results of the solution build.
  3. Create a scripted deployment and undeployment to allow an automated install of the solution.
  4. Deploy the solution to a test environment for system testing.

The preceding checklist makes the following assumptions:

  • A "master build server" is available to perform the build operation.
  • The test environment is a separate set of servers from the build environment with potentially different binding information.
  • For simplicity the checklist assumes that the master build server has the BizTalk Server 2004 runtime installed and that the master build server can be used to perform a simple build validation test before creating a full deployment package.
Build and Deployment Checklist: One-Time Tasks
  1. On the master build server, create location for source files C:\build\source.
  2. On the master build server, create location for binding files C:\build\binding.
  3. On the master build server, create location for binary files, C:\build\bin.
  4. On the master build server, create location for helper binary files, C:\build\bin\helpers.
  5. On the master build server, create location for deployment and undeployment scripts, C:\build\scripts.
  6. Copy into the scripts folder the sample deployment and undeployment scripts and associated VBScript files listed in Table 20 under "Redeployment Scripts."
  7. Ensure that no BizTalk projects use "specify now" ports because these will disallow any binding changes after deployment.
Build and Deployment Checklist: Tasks to Build Binding and Script Files

Create the directory structure:

  1. Empty location for source files, C:\build\source.
  2. Empty location for binding files, C:\build\binding.
  3. Empty location for binary files, C:\build\bin.
  4. Empty location for binary files, C:\build\bin\helpers.

Obtain, update, and test the latest version prior to packaging:

  1. On the master build server, get a local copy of the latest version of the master solution containing all the BizTalk Server 2004 projects to C:\build\source. This copy may come from Visual SourceSafe or from the master source file server. Ensure that any helper or dependent projects are also in the solution and are copied down to the master build server.
  2. Update the assembly version number and file version number to the next version number for all projects that are to be labeled with the current build number. (You can automate this by using the tools described in "Updating Project Version Number and Binding Using Scripts.")
  3. Build the whole solution using a deployment build configuration (as opposed to the development build configuration).
  4. Undeploy any previous versions of the BizTalk Server assemblies from the master build server.
  5. Deploy the newly compiled solution to the clean master build server by using Visual Studio .NET BizTalk Explorer.
  6. Bind the locally deployed solution by using Visual Studio .NET BizTalk Explorer.
  7. Ensure that the binding files are accurate by performing a simple build validation test of the assemblies deployed to the master build server. This test could use a simple test harness or stub, and is carried out to ensure that the configuration is correct before exporting the configuration's binding files.

Collect and build the files needed for deployment:

  1. Copy DLLs from any helper and dependent projects from the bin locations under C:\build\source into c:\build\bin\helpers.
  2. Copy compiled BizTalk Server assembly files from the bin locations under C:\build\source to C:\build\bin.
  3. Use the BizTalk Server Deployment Wizard to export the LOCAL master build binding files for all installed assemblies.
  4. Use the BizTalk Server Deployment Wizard to export the LOCAL master build binding files for any bindings not associated with specific assemblies.
  5. Build an undeployment script based upon the undeployment script template in the C:\build\scripts folder by modifying the "undeploy.cmd" command file.
  6. Test the undeployment script by running the modified undeploy.cmd file and ensuring that it stops and uninstalls the solution.
  7. Build a deployment script based upon the deployment script template in the C:\build\scripts folder by modifying the "deploy.cmd" command file. Ensure that the command file is modified to point to the appropriate files.

Test the deployment files:

  1. Test the deployment script by running the modified deploy.cmd file and ensuring that it installs, enlists, and starts the solution.
  2. Retest the undeployment script by running the modified undeploy.cmd file and ensuring that it stops the orchestrations, and unenlists and uninstalls the solution.

Add additional data required to perform install:

  1. Copy any required test data into C:\build\verify\testdata.
  2. Copy any required test harnesses into C:\build\verify\testharness.
  3. Copy the contents of c:\Build to a version-labeled deployment package, for example, c:\packages\SysInteg_Ver_1.2.3.0.

Modify binding files to target test environment:

  1. Modify the binding files in the package location to produce a set of test binding files that target the test environment (replace local server names with test environment server names, and so on).
  2. All the files required to complete the deployment to the test environment should now be ready under the c:\packages\SysInteg_Ver_1.2.3.0 folder.
  3. Copy the c:\packages\SysInteg_Ver_1.2.3.0 folder to the test server.
  4. Run c:\packages\SysInteg_Ver_1.2.3.0\scripts\deploy.cmd to deploy the solution using the test binding information.

After completing the preceding steps, you can follow with further build and development cycles, reusing rather than re-creating the binding and deployment scripts.

Automated Build Processes

An automated build process is designed to enable a team with a large number of team members, or a large number of code assemblies under development, to efficiently perform a unified build process. The unified build process produces a set of assemblies that can be used for testing the whole integration solution (as opposed to the isolated testing of individual processes on developers' workstations).

An automated build process is heavily dependent upon flexible access to developers' source code. Typically an automated build process integrates with a source management system like Visual SourceSafe.

Automated Build Example

The automated build example described here is a simplified version of a typical build process used in a customer project. It is provided as an example from which a team could develop a fully automated nightly build process with related individual build processes for developers.

The main difference between the example provided here and a completely automated build process is that the full process also needs to contain scripts to manipulate Visual SourceSafe. Scripts can be written (using the Visual SourceSafe COM interfaces) to automate the extraction of the appropriate version of the source for the build required.

The process is described here as an example of the type of steps typically required, because an individual team's requirements will vary. All scripts mentioned in this section are included in the "Automated Build Example" folder of the accompanying samples.

The example assumes that a workstation or server is available as a build server and that the assembly version number is incremented with each build.

Automated Build Process Overview

The objective of the build process is to pull the latest version (or specific labeled version) out of Visual SourceSafe onto a build server, perform a complete compile, and if the compile was successful, produce a deployment kit as outlined above.

To perform an automated build process:

  1. Delete any previous version of source files.
  2. Pull all source files from Visual SourceSafe.
  3. Change build number.
  4. Compile shared DLLs.
  5. Compile process and other DLLs.
  6. Create deployment kit from newly compiled source.
  7. Change any references in source files to match new build number.

The following table shows the folder structure on the example build server.

Table 22 Folder structure on build server

Location Purpose

C:\BuildScript

Contains the scripts that drive the build process.

C:\BuildDist

Location for creating deployment kits.

C:\Build

Contains source files pulled out of Visual SourceSafe.

C:\localSdrive

(shared and mapped as S)

Where common files are built to and referenced via S: (allow full control on share permissions).

Map V: to Visual SourceSafe repository

V: to allow access to Visual SourceSafe repository.

About the Automated Build Process

The following list gives you additional information and hints about the automated build process:

  • The structure of the build server has to reflect the structure under which the developers have developed the solution.
  • In the example there are some shared, common DLLs that are referenced via a mapped drive (S:).
  • The hints contained within the BTPROJ file for DLL references are relative paths for DLLs on the same drive and absolute paths for DLLs on another drive. This can cause issues if you try to compile the solution at a higher level of nested folders that it was originally developed. For example, if you develop the solution three folders deep, the relative path to the BizTalk DLLs would be "..\..\..\Program Files\Microsoft BizTalk Server 2004\". If you tried to compile the solution with folders four deep, it would not find the path to the Program Files directory.
  • The common shared DLLs had a custom Visual Studio build configuration (called SharedBuildConfig) that placed the DLLs on the S: drive.

There are two versions of the build process:

  • A nightly build process that is intended to run on the build server.
  • A local build intended for developers to do a local compile on their workstations.

The main difference between the two processes is that the developer process leaves no trace on Visual SourceSafe whereas the actual build process increments the build number and other numbers.

Automated Build Process Details

The command file that drives the build process is called MasterBuildScript.bat. MasterBuildScript.bat is executed by passing a parameter that indicates whether it should run in build server mode or developer build mode. MasterBuildScript.bat is normally called by using MBS_Log.bat or localMBS_Log.bat, which redirects all output to a log file.

The following steps outline the process:

  1. Set up environment variables dependent on which mode is running (master or local).
  2. Clear out previous builds.
  3. Call the appropriate script to extract source from Visual SourceSafe.
  4. Put references for files such as .snk files back onto the S: drive.
  5. Call the script to compile all projects using devenv.
  6. Check for success indicator (a file called success.txt). If it exists, then build a deployment kit.
  7. Create a new folder structure called "building". Then copy all the required files from the BuildWorkArea to that structure.
  8. Call a script to manipulate the contents of "building" by changing all references in binding files to match the new build number. Then copy this kit to a directory name that is the same as the build number and the "Latest" directory.
  9. Clean up the temporary folders underneath the "Building" folder.
  10. If running on the build server, drop the S: drive and distribution shares before trying to update files located in them to remove any locks.

The following table describes the automated build files.

Table 23 Description of automated build files

File name Description

MasterBuildScript.bat

Main script that executes the build process.

MBS_Log.bat

Calls the MasterBuildScript in build server mode and outputs the results to a log file called MasterBuildScript.log.

LocalMBS_Log.bat

Calls the MasterBuildScript in developer mode and outputs the results to a log file called LocalMasterBuildScript.log.

CopytoBuildTemp.bat

Copies compiled source and other required files for BuildWorkArea structure to deployment kit structure in the "Building" folder.

LocalGet.bat

Called when performing a developer build to get the source code. Could be from Visual SourceSafe or from file system. The sample uses VSS.

LocalBuildNumber.txt

Contains build number used for a developer build.

SetAcl.exe

Used to change the permission on a share.

PostBuildStructure.vbs

Works on Building folders, uses build number to modify binding files, and copies Building to build number directory and Latest.

LocalSetVersion.vbs

Used in developer mode to set the version number of the build to whatever value is in success.txt.

ChangeBindingFiles.vbs

Part of deployment kit, used to change the version number of the assemblies in the binding files to match the build number.

Alternative Automated Build Processes

The functionality of the example automated build could also be achieved by using alternative tools. A team wishing to develop an automated build process could also consider the information in the following table.

Table 24 Alternative automated build tools

Technology Description

BusyBeeBuilder

Provides a powerful, easy-to-use, and extensible .NET build automation tool. It is written with a pipeline architecture that operates plug-ins implementing a simple interface, so it's easy to add your own build steps or use the several that ship with BusyBee. For more information about BusyBee see "BusyBeeBuilder--Extensible Build Automation Tool" on the GotDotNet Web site at http://go.microsoft.com/fwlink/?LinkId=43669.

BuildIt

An automated build tool for Visual Studio .NET.

For more information about BuildIt, see the GotDotNet Workspaces Web site at http://go.microsoft.com/fwlink/?LinkId=43670.

You can download BuildIt from the Microsoft Download Center at http://go.microsoft.com/fwlink/?LinkId=43671.

NAnt

An alternative automated build tool for building .NET projects.

For more information about NAnt go to the NAnt Web site at http://go.microsoft.com/fwlink/?LinkId=43672.

Automated Deployment with BizTalk Server 2004

This section covers how a team developing with BizTalk Server 2004 can achieve an efficient process to collectively automate the deployment of the resulting solution. It focuses on automating the deploying process to allow nightly builds and test processes to be run on a set of test servers. It does not focus on the topic of deployment to a production environment, although some of the techniques may be applicable to this task.

Deploying a "real-world" BizTalk Server 2004 solution typically consists of several related tasks, including:

  • Installing BizTalk Server 2004 assemblies to all servers in the BizTalk Server 2004 group.
  • Generating and installing related BizTalk Server 2004 binding files.
  • Installing shared assemblies (like helper classes) to all servers in the BizTalk Server 2004 group.
  • Installing ASP.NET Web services to all servers in the BizTalk Server 2004 group acting as receive servers.
  • Installing any required test harnesses and test stubs.
  • Installing any required cross-reference seed data.

Typically a deployment also performs operational tasks like enlisting and starting processes (in the right order) and clearing down logs and tracking databases.

BizTalk Server Deployment Toolkit

Earlier sections of this document have described relatively simple approaches to creating a solution structure that includes reusable scripts that allow the creation of simple deployment and undeployment scripts. For a larger development team or a team with a particularly complex or large solution to deploy, these approaches may not be flexible or scalable enough. The deployment toolkit described here is provided as an alternative approach to allow greater control, at the cost of greater complexity to set up the initial processes and scripts.

This section provides an overview of a deployment toolkit developed as a template to enable BizTalk Server 2004 build managers to construct a scripted, automated deployment process. This toolkit is installed in the samples folder called BizTalk Server 2004 Deployment Toolkit.

The BizTalk Server 2004 deployment toolkit provides a structure for packaging a BizTalk solution—assemblies, binding files, helper classes, Web services, and so on—to allow a person with no previous BizTalk knowledge to configure and deploy a complete BizTalk solution. This kit also covers undeployment of most stages to leave a clean system.

All scripts and other files mentioned in this section are included in the accompanying samples in the BizTalk Server 2004 Deployment Toolkit folder.

This deployment kit was initially developed to support the rollout of a large BizTalk solution into the following environments:

  • Development environment
  • System test environment
  • Integration test environment
  • Production environment

The process has to be repeatable for a given configuration and needs to be done by users who are not BizTalk aware. The kit uses scripts to configure and deploy the runtime environment without any reliance on Visual Studio being deployed on production servers.

It is our hope that through this guide, solution designers have gained an understanding of the constituent parts of a BizTalk solution and the possible project structures used to develop BizTalk solutions.

We also trust that developers have gained a better understanding of the process of developing a BizTalk solution in a team.

Project managers will also have gained an overview of the main project phases and gained a better understanding of the typical tasks necessary when planning a BizTalk Server 2004 team development.

The following resources may be of use to BizTalk Server designers, developers, and project managers:

  • MSDN Integration Patterns. Integration Patterns explains how patterns were used to design and build an integration architecture within a representative customer scenario. The guide includes a catalog of 18 common integration patterns including implementations that use the Microsoft platform.
  • Microsoft BizTalk Server white papers. White papers covering the design, development, and deployment of BizTalk Server 2004. In addition to the information at MSDN, you can find Microsoft BizTalk Server 2004 white papers at the BizTalk Server 2004 Web site at http://go.microsoft.com/fwlink/?LinkId=43978.
  • Team Development with Visual Studio .NET and Visual SourceSafe. This guide provides guidance and recommendations to enable you to set up a team development environment and work successfully within it.
  • ThoughtWorks Test-Driven Development in Integration Projects. Paper covering approaches for testing integration developments. You can find this white paper at http://go.microsoft.com/fwlink/?LinkId=43979.

The following steps provide a sample structure for creating a partitioned single solution that integrates with Visual SourceSafe and holds the non-BizTalk project entities that are required (for example, test data or scripts).

This work example assumes that you are following the partitioned single solution model for larger or more complex solutions. You can create the single solution model by following the same steps and excluding the "Creating a partitioned solution" section.

Use the following steps as a sample and not as a definitive guide. They are presented here to assist developers new to team development with BizTalk Server 2004 and Visual SourceSafe with a "quick start" approach. This approach allows the team to familiarize themselves with the partitioned single solution approach and evaluate which Visual SourceSafe approach will best suit the team needs.

Use the following steps and information to create an example Visual Studio .NET solution structure that integrates with Visual SourceSafe.

  • Create the local root folder. First, create the top-level folder under which all source is stored on developer workstations. In this example, the folder is called SysInteg2004. Create the folder c:\SysInteg2004 on the workstations that you are using to create the initial project structure.
  • Master solution and Visual SourceSafe restrictions. It is important to note that all Visual Studio .NET projects that are included within a master Visual Studio .NET solution and additionally integrated with Visual SourceSafe must be created in the file system folder that contains the master solution file. The _Master_Solution folder created below serves this purpose. For more information about team development, see Team Development with Visual Studio .NET and Visual SourceSafe.
  • Create the master new solution from within Visual Studio .NET. To create the empty master solution that you will use as the container for all projects in the build process and Visual SourceSafe control, complete the following steps:
    1. Open Visual Studio .NET.
    2. On the File menu, click New Blank Solution. Then create the new master solution in a folder under C:\SysInteg2004, for example:
      c:\SysInteg2004\_Master_Solution
      
    3. Make sure that the Create directory for solution check box is selected. This option is available under the More tab, as shown in the following figure.
      Figure 14 New Project dialog box
      n/a

      ms942199.note(en-US,BTS.10).gifNote
      The leading underscore naming convention is used here to denote a folder containing a solution as opposed to a folder containing an actual project. This notation can be helpful if you are creating many partitioned solution folders alongside of project folders.

  • Add the solution to source control. After you create the master solution, you must add it to Visual SourceSafe.
    1. In Visual Studio .NET, in the Add to SourceSafe Project dialog box, right-click the solution and then click Add Solution to Source Control.
    2. In the Visual SourceSafe Add to SourceSafe Project dialog box, overwrite the default project listed in the text box with SysInteg2004.
    3. Click OK to create the top level of your Visual SourceSafe tree.
  • Create a new BizTalk Server 2004 project within the master solution. Perform the following steps to add a new project under the master solution. You can use this project for both partitioned and single solution models.
    1. In Visual Studio .NET Solution Explorer, right-click the solution, point to Add, and then click New Project to add a new BizTalk Server 2004 project.
      Note that the new project directory is located underneath the master solution (for example, C:\SysInteg2004\_Master_Solution\MonthlyBillingRun).
    2. Check out the master solution if required.
    3. Add additional BizTalk Server 2004 projects using the preceding two steps.
    4. Check in the master solution. This ensures that all the files are written to the Visual SourceSafe server before any further operations take place.
  • Add shared projects to the master solution. You add shared Visual Studio .NET projects like C# or Visual Basic .NET helper classes that are included in the build process of the project to the master solution in the same way.
    To add a shared project to the master solution:
    1. In Visual Studio .NET, open the master solution.
    2. In Solution Explorer, right-click the solution, point to Add, and then click New Project to add the new helper project.
    3. Click Browse to create a shared folder structure under the master solution.
    4. In the Project Location dialog box, create a folder structure to hold the shared project (for example, c:\SysInteg2004\_Master_Solution\shared\test).
      The project is added under the newly created shared section of the solution.
    5. Repeat this process for other shared projects.
      The following figure shows the Add New Project dialog box.
      Figure 15 Add New Project dialog box
      n/a

  • Add non-project files to the structure. When developing a complete solution, you might create additional files not directly related to the Visual Studio .NET project, for example, test data and deployment or testing scripts. It can be beneficial to the project to manage these files by using Visual Studio .NET because this allows the following:
    • You can use Visual Studio .NET to edit, view, and manipulate the contents of the files. This can be useful, for example, when copying and pasting test data between sample files.
    • All source-control operations are managed by using Visual Studio .NET (rather than the Visual SourceSafe explorer), which is the recommended approach.
  • Managing project-specific files. If the files are related to a specific Visual Studio .NET BizTalk project, then you can include these files in the BizTalk project and manage them in the same way as all other project files by performing the following steps.
    To include a file or folder into an existing Visual Studio .NET project:
    1. In Solution Explorer, click the "Show All Files" icon,
    2. Right-click the folder or file to include in the solution, and then click Include In Project,
    The following figure shows the dialog box.
    Figure 16 Solution Explorer showing Include In Project
    Including files in a project

  • Managing shared or generic files. If the non-project files are not associated with a specific process (for example, deployment scripts for the whole solution), they should not be associated with a process-specific project. To help manage these entities, create an empty C# Visual Studio .NET project to act as a container for these files (Empty Project is available as a C# template in the New Project dialog box). Add this project to the master solution (under the shared path as detailed earlier in "Add shared projects to the master solution"). Having created the project use the "Include In Project" functionality to add non-project files to the container project.
    When the master solution is used to configure the build process, projects containing non-project files should be excluded from the build process to avoid build errors relating to the fact that the project is not a valid C# project.
    See "Version controlling non-BizTalk Server project files" for more details on how to add non-project files to a project.
  • Create a common location for file dependencies. It is useful to create a subfolder to hold the pre-compiled binaries for the project, that is, DLLs that are not produced from Visual Studio projects that the team has control over. For example:
    c:\SysInteg2004\bin
    
  • Create a partitioned solution for customer operations only. As discussed in this document, the partitioned single solution provides benefits when working with a complex or large solution. The following steps show how to partition the master solution to create a partitioned solution that holds just the two customer projects in our scenario:
    1. Ensure that the newly created master solution has been checked in (this is to ensure that Visual SourceSafe has copies of the projects about to be included in the new partitioned solution).
    2. Open Visual Studio .NET.
    3. On the File menu, click New Blank Solution. Then create the new project in a folder under SysInteg2004, for example, c:\SysInteg2004\_Customer_Solution.
      Again the leading underscore "_" is used to denote the fact that this project contains a solution rather than a project.
      The following figure shows the New Project dialog box.
      Figure 17 New Project dialog box
      Creating a partitioned solution

    4. Add the solution to source control by right-clicking the solution, and then choosing Add to Source Control.
    5. In the Add to SourceSafe Project dialog box, select the SysInteg2004 folder as the location to create the project, as shown in the following figure.
    6. Clear the Project text box.
    7. Click OK.
      Figure 18 Dialog box with Project text box cleared
      ms942199.Local_-716283098_figure18(en-US,BTS.10).gif

  • Add projects to the partitioned solution. Now that you have created the new solution, the next step is to add the projects that belong under the customer solution. In our sample this means adding the customer projects.
    To add the first customer project (AmendCustomerDetails) to _Customer_Solution:
    1. On the File menu, point to Source Control, and then click Add Project from Source Control.
      Figure 19 Add Project from Source Control
      Adding a project from source control

    2. In the Create a local project from SourceSafe dialog box, select the AmendCustomerDetails project, and ensure that the path in the Create a new project in the folder text box is pointing to the folder that holds that project on the file system, as shown in the following figure.
      Figure 20 Pointing to correct folder
      Adding a project to a solution and confirming the

    3. Check out the solution if required.
  • Add a project to the solution and confirm the project location. You may want to add the second customer project to the solution.
    To add the second customer project (CreateNewCustomer) to _Customer_Solution:
    1. On the File menu, point to Source Control, and then click Add Project from Source Control.
    2. In the Create a local project from SourceSafe dialog box, Select the CreateNewCustomer project.
      ms942199.note(en-US,BTS.10).gifNote
      By default, the Create a new project in the folder text box is pointing to the previous customer folder and needs to be changed to the second customer project on the file system.

      The following figure highlights the CreateNewCustomer project that is to be added, and highlights the path that must be changed to contain the path to the CreateNewCustomer project. If this path is not updated, an inconsistent folder structure will be created.
      Figure 21 Path for new project
      not available

    This partitioned solution can now be used by the developer who is responsible for developing the customer areas of the solution, without requiring the developer to retrieve and store the master solution on a local workstation.
  • Re-create the solution structure on a new workstation. The following steps show how the Visual SourceSafe structures created in the previous steps allow a new developer to the team (or a developer with a new workstation) to quickly retrieve the necessary solution structure.
    These steps assume a different Visual SourceSafe user from the one used in the previous steps and a clean workstation file system.
    To retrieve the folder structures required for the solutions:
    • Create the top-level folder and the solutions required (in this case _Master_Solution and _Customer_Solution). Create the following folders:
      c:\SysInteg2004
      c:\SysInteg2004\_Master_Solution
      c:\SysInteg2004\_Customer_Solution
      
  • Retrieve the solution onto the developer workstation. In this scenario, the new developer needs to work on the customer solution only. To obtain the necessary solutions and projects:
    1. Open Visual Studio .NET.
    2. On the File menu, point to Source Control, and then click Open From Source Control.
    3. In the Visual SourceSafe dialog box, select the _Customer_Solution solution and select the project path c:\SysInteg2004\_Customer_Solution.
    4. Click OK.
    The customer solution and the appropriate projects will be retrieved from the Visual SourceSafe repository. The files will be located on the local hard disk in the same folder structure as on the hard disk of the workstation used to create the Visual SourceSafe structures. In this way build, deploy, and testing scripts can be shared across all workstations without modification.
  • Prepare a master build server. A build of the complete solution is typically created by using a solution that contains all the projects. In this way Visual Studio .NET can determine the build order for the solution and ensure that all dependent DLLs are built and updated. If you are using a separate workstation or server to perform the master build process, you can use the preceding steps to retrieve the _Master_Solution as the basis of setting up the master build server.

Technical Contributors

Jason Birth, Jonathan Bonnick, Mariusz Borsa, Barry Boudreau, Scott Colestock (Trace Ventures, LLC), Akhtar Hossain, Dave Howe, Brett R. Johnson, Stephen Kaufman, Suresh Kumar (Shell), Kevin Lam, Erik Leaseburg, Bob Leithiser, Brian Loesgen, Jonathan Moons, Bhushan Nene, John Plummer, Mandar Samant, Kevin Smith, Rob Steel, Jody Sweeton (Shell), Bill Ticehurst, Marty Wasznicky, Kevin Williams, Scott Woodgate, and Ruslan Yakushev.

Show: