BizTalk Server 2006: Understanding BizTalk Server Application Deployment
Summary: This document discusses improvements made to BizTalk Server. Enhanced capabilities and streamlined processes are illustrated using hypothetical business situations.
BizTalk Server is Microsoft’s premier server for building solutions for business process and integration. BizTalk Server 2006, the fourth major release of the product, builds on the innovation and success introduced by the previous three releases: BizTalk Server 2000, BizTalk Server 2002, and BizTalk Server 2004. The 2006 release includes a number of key new developer and administrator tool improvements focused on improving the deployment and manageability of BizTalk Server solution.
In BizTalk Server 2006, application deployment has not changed dramatically under the covers, but many tools enhancements have been undertaken to ensure this process is much more intuitive and less error-prone for users.
This section will call attention to specific features which have been added or improved in the Application Deployment space which should be especially helpful to those working often with BizTalk Server. The audience for this section is Developers and IT Professionals new or veteran to BizTalk Server.
The first major enhancement in this area is the introduction of the application concept. This is really just a logical container for application artifacts, allowing you to bucket related components. However, the effect is a streamlining of many of your everyday tasks. By being able to deploy, manage, start/stop, and troubleshoot all at the application level, there becomes less confusion and less risk of errors for users.
For example, a developer can specify an application to deploy to by editing the "Application Name" property of a Microsoft Visual Studio project (see Figure 2). By doing so, their artifacts will be deployed to the said application. This creates a common vocabulary between the Developer and the IT Professional when they speak of "an application."
For the IT Pro, they can use the Administrative MMC Console to configure and start an entire application. Performing these tasks at the application level ensures that no stone is left unturned. Once running, the application concept allows the IT Pro to manage and troubleshoot selectively at the application level. In other words, from a holistic view of their entire system, users can choose to drill down into errors coming from any one specific application.
Figure 1: The BizTalk Server 2006 Administration Console
This concept also assists greatly with the tasks related to Developer deployment as well as IT Pro deployment and staging, segueing into our next sections.
Developers typically deploy to their own local BizTalk server to sanity test the solution still in progress. The developer deployment and re-deployment process has been streamlined considerably since BizTalk Server 2004.
First off, the aforementioned application concept allows different projects to deploy into logical application containers, bucketing like artifacts. This allows users to easily enter the Administration Console after deploying from within Visual Studio and create ports and bindings for this application.
Once deployed, the developer typically tests, makes changes, and redeploys. In BizTalk Server 2004 this required a lot of manual intervention on the part of the developer to stop and tear down the dependent components in the correct order, redeploy all of the artifacts, restart host instances, and re-create/start all of the dependent artifacts (e.g. ports, orchestration enlistments, etc.). With BizTalk Server 2006, the developer experience is much more streamlined and less susceptible to user error. After making changes to the project already deployed (say fixing an orchestration or editing a map) and with the Redeploy = True and Restart Host Instances = True properties for the project set, developers can simply Right-click … Deploy the solution and voila! This removes many formerly manual steps.
Figure 2: BizTalk Project Deployment Properties
But application deployment does not just start and stop with the developer. The next section explains more about the feature advances which will help the IT Professional.
Deployment and Staging
Deployment is the logistical distribution of application artifacts to ensure all necessary components are available to the systems that require them. Staging, more specifically, refers to the moving of application artifacts from environment to environment, say from development to a test environment, or from staging to production to undergo different levels of testing at each stage.
Generally, deployment has been improved with the introduction of the application concept. Since an application typically contains all of the necessary components that its parts depend on, this logical container can be leveraged to help roll-out artifacts to other machines added to the group or for staging (when transporting an application to another environment).
Naturally, the application container contains BizTalk Server assemblies which are deployed to it by the Visual Studio environment, as explained earlier. Further, users can manually add BizTalk Server assemblies to the application, or move BizTalk Server assemblies from other applications. Likewise, they can also add non-BizTalk Server assemblies to the application. The application may also contain receive ports, receive locations, send ports, property tracking settings, and role links, to name a few. Implicitly, the application also contains all of the bindings that are represented by their current settings. And other BizTalk Server artifacts can also be added to applications, such as BRE (rules) policies and BAM definition files.
But in an effort to create a more comprehensive application container, we also provide the ability for users to add additional (typically non-BizTalk Server) artifacts to the application under the Resources node. Scripts are also a welcome type of resource which can be chosen to run during different application deployment operations.
Now that an application contains all of the necessary artifacts, the actual deployment process comes in. BizTalk Server 2006 employs the help of MSIs to actually perform deployment and staging. The following section covers this process.
Working with MSIs
Microsoft Windows Installer (MSI) technology is used to assist with application deployment tasks. Why use an MSI? MSIs are a standard way for applications to be installed onto Microsoft operating systems. They create registration entries in the Add/Remove Programs list, accelerate deployment by automating deployment of artifacts and their dependencies in the correct order, and also have many additional capabilities which will be described.
With BizTalk Server 2006, users have the option of exporting an application to an MSI file, which serializes all of the application artifacts into this package. This can be done from the Administration Console or by using BTSTask.exe from the command line.
Thereafter, several operations can be done with this MSI, explained below.
IMPORT – This is a group level deploy (once per group).
This operation will import bindings resident in the MSI package, deploy all BizTalk Server assemblies to the Management Database for the group, run scripts specified to run at import time, et cetera. This is done by opening an MMC and doing an import operation of the MSI (or via an import operation from the BTSTASK command line).
INSTALLATION – This is a per box deploy (n times per group ).
This GAC’s all BizTalk Server assemblies and dependency assemblies so that this machine has all of the binaries it needs for runtime. This operation can also roll out related web services which might be part of the solution (e.g. orchestrations published as web services). Further, this operation can be used to apply machine-specific changes, such as pre-creating MSMQ queues, creating FILE drop folder structures and permissions, et cetera, which can be done with the help of scripts. This operation is done by double-clicking on the MSI file itself. Instead of being done once per group, it must typically be done on each BizTalk machine that is a member of the group.
Depending on what type of roll-out you are doing, you may have to do one or both of the above operations.
Figure 3: Importing an Application MSI
One such scenario, Scenario A, might be adding an additional server to the BizTalk group. You may want to use MSIs to help with this. Exporting the application as an MSI and installing it on the new machine would probably be sufficient to prepare this machine for runtime, provided the application was already imported previously to the group and the server has already been JOINed to the group in Installation/Configuration.
However, Scenario B might include setting up a brand new environment with the solution, in which case the application and its contents must not only be installed on all runtime machines in the new group, but also registered with the Management Database for the group so that the application is created, the bindings are applied, the BizTalk Server assemblies are deployed, et cetera. This would necessitate running the MSI installation (double-clicking the package) on all n servers, but also importing the MSI once for the group.
Using Scripts with MSIs
Scripts can be helpful when customization to the machine (installation operation) or to the group (import operation) becomes necessary.
The documentation article entitled, "Using Pre- and Post-processing Scripts to Customize Application Deployment" in our core documentation, explains how scripts can be added as resources and how (and when) they are called by the MSI.
The "long story short" is that you can add scripts to run as pre-processing or post-processing scripts. However, you will have to include logic in your scripts to check environment variables to determine which context the script is executing in (an import, installation, or uninstallation) and process accordingly.
There are some additional features in BizTalk Server 2006 which can better facilitate staging. Clearly, exporting applications as MSIs can greatly expedite the process. But even beyond this, we have added some new capabilities which make transporting solutions from environment to environment even easier.
For example, every application contains a "Resources" node which is a general catch-all for application components. One of the Resource types is called "BiztalkBinding". This allows users to add additional bindings files to an MSI. For each bindings file that is added, the user must specify the environment name (text field) which the bindings should be applied to. Then on import, the user will be asked to choose the environment name to which they are presently importing. If these match, then the bindings file is applied to the target group. Users can also choose to specify no environment name for the added bindings file which will cause this set of bindings to be applied to all environments unconditionally.
Figure 4: Adding Environment-specific Bindings
As explained earlier, the application implicitly has a set of bindings by default, since ports and locations exist and the transport properties are already configured. These settings will be serialized into an XML file, included in the MSI file, and tagged as belonging to the "Default" environment. At import time, this "Default" environment can be chosen by the user, or circumvented by selecting another.
Often times transports, URIs, or endpoints can change from environment to environment, and this feature allows an IT Pro, or developer, to create one package which contains all of the transport properties for all environments which it requires testing in. One monolithic package can then be distributed around to service them all. Some customers will know all of these bindings a priori and can take advantage of this feature. Others, however, may not know these details upfront at the time of MSI creation and will have to update bindings manually after importing the MSI.
For complete information on all of the new deployment features in BizTalk Server 2006, please consult the core BizTalk Server documentation included with the product and updated online at MSDN. The article entitled "Deploying and Managing BizTalk Applications" is a fine place to begin.
Now that you understand how to get applications deployed and running in production, at some point they’ll probably need to be upgraded. Perhaps a partner’s schema will change so your schema and map will have to be changed accordingly. Or maybe the business will require a new auditing API to be called during one part of the business process’s execution. And for as good as our BizTalk Server developers are, there are always code defects in application code that will need to be fixed.
So how does BizTalk Server accommodate these application upgrades? Well, there are a couple of different ways to upgrade your running applications, and as always, different points to consider.
The audience for this section is primarily IT Professionals who are fairly experienced with BizTalk Server.
One scenario we’ll deem the "Simple Upgrade Scenario". This scenario requires no downtime, requires no code changes, and can be performed by an IT Pro or Business Analyst (BA) single-handedly.
Some examples include:
An additional partner requests access to all orders of a certain type
A business rule changes
A queried web server will be phased out and replaced with a new one
Changes in this scenario are typically as simple as adding an additional send port to a send port group, adding an additional send port with appropriate subscription filters, changing the URI of a send port, updating a rule in the BRE, et cetera.
Figure 5: Changing Endpoint or Transport Properties
However, there may be more involved cases of application upgrade.
One such scenario is called the "Patching Scenario" in which existing application binaries in production must be edited and swapped with updated ones.
Typical examples include:
Patching an orchestration with a code change
Changing a schema
Updating a map
In this case, the customer scenario must meet the following conditions:
System downtime can be scheduled for application upgrade
Customer does not have long-running business processes
Dehydrated or suspended instances can be quickly resumed and completed, or alternately terminated
If the customer is updating an orchestration, say, and meets these criteria, the following steps might be their typical marching orders.
First, the application binaries will be updated and recompiled with name and version unchanged. Then, downtime must be scheduled and new instances should be prevented from starting up, that is, publication to the MessageBox should be halted temporarily by disabling endpoints. During this period, any running instances will have to be stopped and unenlisted, which requires all dehydrated or suspended instances to be either manually resumed and completed, or terminated. After service instances are in the stopped and unenlisted state, the new application binaries can be deployed.
To deploy the updated application binaries, first the group’s Management Database should be updated by performing a Deploy operation using the overwrite flag (see Figure 6). Second, each and every server in the group (typically the case, but sometimes only a subset of the total servers may have to be patched if host processing groups are utilized) must be updated by GAC’ing the changed assembly or using an MSI as described in the "Leveraging MSIs for Upgrade Scenarios" section. Finally, all BizTalk host instances should be restarted. At this point, the new orchestrations can be re-enlisted and started, and message publication can be resumed.
Figure 6: Patching a BizTalk Server Assembly
However, some customers may have more stringent requirements for upgrade. For example, they may not be able to schedule downtime or may have very long-running instances which cannot be terminated. In these cases, side-by-side versioning may be required.
Side-by-side (SxS) Versioning
This scenario, entitled "Side-by-side Versioning," allows two versions of the same application artifacts to be running side-by-side. The .NET runtime inherently allows for same-named but different versioned assemblies to be deployed and running. BizTalk Server also allows for this, although some discretion is advised.
Figure 7: Versioned BizTalk Server Assemblies
Some BizTalk artifacts, such as maps, are chosen by fully-qualified strong name, or FQSN, which means the bindings are mindful of the version used. Making a code change to the map, upping the version number (though only major and minor build numbers should be changed), compiling, and deploying the additional assembly to the group (and all machines) would allow users to simply select the new map for inbound or outbound mapping (see Figure 8). However, calling maps from orchestration may require code changes to the orchestration itself if the map reference is hard-coded.
Figure 8: Versioned BizTalk Maps
Making changes to orchestrations can be a bit more involved. If you have short-lived orchestrations, then the “Patching Scenario”, previously described, may be sufficient. But if you have long-running orchestrations or cannot terminate existing instances, then side-by-side versioning will be your only alternative. Typically, the SxS story for orchestrations would go something like this.
Contoso Mortgage Company needs to update their orchestration, but has many existing orchestration instances in flight that aren’t expected to terminate for weeks or months. They’d like for their existing instances to culminate on the live DLLs, but have new instances start up on the revved assemblies.
To do this, they begin by having the developer increase the orchestration’s version number (again, major and minor builds only) and make code changes to the orchestration. Next, they’ll deploy this new assembly to the group and GAC it on all runtime machines. Then the customer has the option to create new receive ports and locations for the new version or use existing ports.
If the former is chosen, simply binding to the new ports and enlisting/starting the new artifacts will probably be sufficient. Creating new receive locations and ports is usually the preferred approach, especially since if your scenario uses long running orchestrations where a number of correlating receives are still needing to be processed, this may not allow you to reuse existing receive ports or perform unenlistment. If you create new ports, be sure that it’s possible for your backend and partner systems to handle this change. If not, you’ll have to wait for all long running instances to culminate before upgrading.
If the latter is chosen, some additional steps apply. The customer will have to (a) bind the new version of the orchestration to existing ports, (b) unenlist (but do not stop) the old orchestration version, and (c) enlist and start the new orchestration version. The documentation includes a script which help you to do steps (b) and (c) in one transaction so that messages are not missing subscriptions in between manual clicking.
The end result is that future message publications will be directed to the new orchestration version, and already running orchestration instances will complete on the older version. The product documentation should be consulted and followed closely for this kind of procedure, or for scenarios that depart from the "norm", such as orchestration use of role links, direct binding, Specify Now binding, BAM activities, et cetera.
Upgrading pipelines has several alternatives as well. The simple solution is to simply choose the newly deployed pipeline version in the send port or receive location. This will replace the old pipeline with the new one. However, if true side-by-side functionality is required for backwards-compatibility purposes, then new send ports and receive locations will have to be created and bound to with the new pipeline version specified.
Finally, updating schemas also has some peculiarities. Side-by-side versioning is absolutely allowed, but the schema resolution behavior for the XML disassembler should be examined closely (see the "Schema Resolution in Pipeline Components" article). In certain cases, customers may want to hard-code references in the pipeline disassembler properties to specific versions of the schemas to avoid the dynamic resolution behavior. This will allow for more “pure” side-by-side scenarios. Customers should also be mindful of the way in which their schemas are used. Changes to a schema assembly may also require making changes to a map or other schemas which may reference this assembly.
Leveraging MSIs for Upgrade Scenarios
Upgrading applications is typically a deliberate and precise operation in production. Because of this, following a manual checklist is advised. However, certain steps may be streamlined by using MSIs.
MSIs are a great way to wrap up your application artifacts into a distributable package. This may help when rolling out updated DLLs to multiple runtime boxes or assist with the group-level deploy. Precaution should be used when creating the MSIs to exclude all other unchanged resources and bindings from the package.
If you are conducting a "Patching Scenario," stopping/unenlisting/re-enlisting/starting steps will need to be done manually before and after Importing/Installing the MSI. Similarly, numerous steps outlined previously will have to be performed in the "Side-by-side Versioning Scenario" before and after using the MSI.
In general, application upgrade is a cerebral activity and any change that you’ll ever make to your production environment should always be well tested and practiced in a pre-production environment. Having a non-production environment which mirrors the live environment will allow you to dry-run this and other production changes.
Needless to say, you should also carefully consult the product documentation before making any changes to your production environment, but this is especially true in the case of application upgrading. For more information on this topic, be sure to check out the documentation article entitled "Managing Artifacts".
New features in BizTalk Server 2006 greatly streamline the experience of deploying and staging BizTalk solutions for both Developers and IT professionals, decreasing the time to solution and improving its manageability. Once applications are up and running in production, upgrading these applications requires careful planning, consideration, and attention to product documentation.