BizTalk Server Business Rules Framework
For the latest information, please see http://www.microsoft.com/biztalk
On This Page
Business rules guide the decision-making behavior of all business processes—they are the conditional criteria against which factual variables are evaluated to determine an appropriate business action. In conventional application design, business rules are coded as procedural implementation methods. Embedding business rules within applications has always been problematic because it compromises the effectiveness and versatility of the application, and consequently the business process the application is meant to enable. The limitations and inefficiencies of this development approach appear at every stage of the application’s development life cycle:
The development process requires a translation step to convert rule logic detailed in a functional specification to procedural implementation code. The rules include business facts, conditional statements, and so on.
When converted to procedural code, the semantics of the rule logic are no longer accessible to the knowledge workers or business analysts responsible for the integrity of the business process.
Embedding business rules in implementation code creates unwarranted codependencies with procedural functions. Embedded business rules do not function independently and cleanly. Consequently, modeling, testing, and modifying the rule logic is complicated.
Embedding business rules requires the developer to create or implement an inference engine to match facts to rules and connect complex rule sets. When facts are introduced into a business process, the application must match rule conditions to the relevant facts, evaluate them, and trigger an action. If the facts are few and the evaluation criteria simple, the logic and required programming effort is uncomplicated. If, however, the facts are abundant and the evaluation criteria are complex, the required programming effort and logic complexity increases dramatically.
Most modifications to a business process life cycle pertain to changes in business rules (as opposed to technology-related modifications). Because conventional applications embed business rules in opaque procedural code, the rules cannot easily be accessed or modified without disrupting running processes.
Despite these limitations, the absence of a better alternative has allowed this methodology to become widely accepted. Until recently, no programmatic development environment existed that separated business rules from implementation methods, preserved rule comprehensibility, and enabled dynamic binding of rules to other functions.
Programming languages remain syntactic, abbreviated, and procedural, as opposed to semantic, expansive, and declarative. Consequently, most programming languages still resemble an abstract representation of machine instructions, or “code.” This is a legacy from the time when computers were constrained by limited computing resources and required extremely succinct programs to make them work optimally—or even at all. In fact, programs were so concise that a program written in assembly language could only be understood by the computer for which it was written and the programmer who wrote it. Today, we have an abundance of raw computing power, but the legacy of resource conservation has had a permanent effect on the evolution of programming languages.
With their structure bound to a computing implementation model, programming languages are not suitable for describing the meaning and function of complex entities and events (processes). As a result, these entities and events must first be modeled using a different methodology. In conventional development, this modeling occurs in the complex and highly interpretive design and specification process: business analysts develop a specification, and programmers translate the specification into implementation code.
Then, after a specification is accurately translated into procedural code, another issue typically presents itself: the behavior of the code is less predictable than the behavior of the process it models. Consequently, when software is operational, any subsequent modifications to the underlying code are discouraged. This is the reason why so many business processes revolve around the limitations of software.
The difficulty of implementing business rules as procedural code to automate a process is just one example of the limitations of current development methodologies. Because business rules play such a central role in defining business models and processes, any attempt to modify or enhance the way a rule is deployed offers a powerful reminder of the need for an agile business infrastructure.
The solution requires a conceptual reorientation within a new development environment—one whose methodology is based on exposing the meaning and function of information and implementation methods, and facilitating the loose coupling of any application components both at design and run time.
Such a development paradigm has emerged, and it has demonstrated the ability to radically improve the efficiencies of creating, modifying, extending, and repurposing solutions for enterprise application integration, process automation, and trading partner interchanges. The Service Oriented Architecture (SOA) paradigm has redefined the concept of an application. No longer an opaque, procedural implementation mechanism, an application is an orchestrated sequence of messaging, routing, processing, and transformation events where both message content and the functional components that operate on the message are exposed using XML technologies. XML-based development and deployment platforms that facilitate the SOA paradigm are highly compelling because they alleviate significant development and life cycle overhead and enable the extension and reuse of components and entire applications to an unprecedented extent.
Applications that rely on sophisticated, constantly evolving business rules stand to benefit substantially from this paradigm. It has long been recognized that isolating business rules entirely from procedural code, or any process implementation mechanism would dramatically improve a business’ ability to manage and adapt their business processes in response to new requirements or business conditions. Consequently, isolating, exposing, and publishing business rule sets as services that can be accessed by any application or process provides one of the most compelling value propositions for the Services Oriented Architecture paradigm.
Recognizing this, Microsoft® has introduced the Business Rules Framework in BizTalk® Server 2004, its XML-based development and run-time environment. This paper examines how business rule “policies” are created and deployed in BizTalk Server 2004, and demonstrates how a development environment that enables business rules to function as transparent services without implementation dependencies can realize the potential of business rules to drive business agility.
BizTalk Server Business Rules Framework
The Business Rules Framework is a stand-alone application that consists of a number of modules, support components, and tools. The primary modules include the Business Rules Composer for constructing policies, the Rules Engine Deployment Wizard for deploying policies created in the Business Rules Composer, and the Run-Time Rule Engine that executes policies on behalf of a host application.
Following is a list of the features and functions each of the modules provide:
Business Rules Composer
The Business Rules Composer employs a graphical user interface that enables developers, business analysts, and administrators to do the following:
Create versions of vocabularies
Create vocabulary definitions and bind them to data sources, including XML schema elements, database elements, and .NET class members
Include vocabulary definitions in rules (Note: rules can be created with or without user-defined vocabularies)
Compose policy versions from a logical grouping of rules
Publish vocabularies, and publish and deploy policies
Rules Engine Deployment Wizard
The Rules Engine Deployment Wizard provides an easy and efficient way to perform the following:
Export a version of a policy or vocabulary from a SQL rule store to an XML file store
Import a version of a policy or vocabulary from an XML file store to a SQL rule store
Deploy a version of a policy to make it available to the update service and the rule engine run-time. The wizard deploys a policy by marking it as “deployed” in the database
Undeploy a version of a policy from a production SQL rule store to make the policy unavailable for use by a rule-based application
Run-Time Rule Engine
The Run-Time Rule Engine performs the following functions:
Processes rules created as declarative statements without procedural instructions
Groups all the rules that apply to a business process together (a policy) without having to arrange them in order of, or define them in terms of, relationships, contingencies, or dependencies
Resolves rule conflicts arising from the simultaneous presence of multiple facts and conditions
Supports forward chaining of rules that involves reevaluating rules in the executing policy based on changes to the fact base (addition/removal of facts or changes in state/data of existing facts). Forward chaining does not inherently span policies
Component Processes and Terminology
A vocabulary encompasses the user-defined names for facts used in rule conditions and actions. Vocabulary definitions make rules easier to read, understand, and share in a particular business domain. For example, the source location for a fact might be a certain column in a certain row in a certain database, represented as a SQL query. Instead of employing the SQL query (an abstract procedural statement) in a rule, a vocabulary definition can be associated with the query using a name understood by the relevant parties in the development and deployment process. The vocabulary item can then be used in any number of rules, and the rule engine will be able to execute the corresponding query. If a vocabulary is not used, the user will see a binding to the DB-Table-Column.
Vocabulary items are stamped with a version and published in a rule store. This guarantees that the definitions in the vocabulary are immutable in order to preserve referential integrity. This means that any policies that employ the vocabulary item will not fail because of changes to the underlying vocabulary. Multiple vocabulary items can refer to the same underlying sources. Vocabularies also allow you to do the following:
Define constraints: constants, sets and ranges that the composer will use to validate and constrain rule parameters
Construct “macros” where a vocabulary may wrap other vocabularies and/or define bindings and binding parameters. For example, a vocabulary definition might refer to a .NET method that takes two parameters. As part of the definition, one parameter may be bound to the value in an XML document and the other may be defined by the user at design-time but limited to the valid values defined in another definition that defines a “set.”
Define a library of bindings that prevents the user from having to work with many different data sources (schema definitions, DBs, assemblies) in building rules.
Business rules are statements that govern the conduct of business processes. The components of a business rule are facts, conditions, and actions. A rule evaluates a fact against a condition. If the fact matches a required condition, then one or more actions will be initiated by the rule engine.
The following rule determines whether a transaction will be conducted by applying business logic and comparing two monetary values—an input amount and available funds (the facts to be evaluated):
IF amount is less than or equal to available funds THEN conduct transaction and print receipt
A condition is a true/false (Boolean) expression that consists of one or more predicates applied to facts. In the preceding example, the predicate “is less than or equal to” is applied to the facts “amount and available funds.” This condition will always evaluate to either true or false. Predicates can be combined with the logical connectives AND, OR, and NOT to form a logical expression that can be potentially quite large, but will always evaluate to either true or false. Actions are the functional consequences of condition evaluation. If a rule condition is met, a corresponding action or multiple actions will be initiated. In the preceding example, “conduct transaction” and “print receipt” are actions that are carried out when, and only when, the condition is true.
Policies are physical rule sets with each policy version existing as a discrete artifact with a concrete representation in the DB. Policy versions can be exported to a stand-alone XML file. A policy is created, tested, and subsequently published and deployed to a production environment. A policy can contain an arbitrarily large set of rules and is deployed by embedding it (where it becomes a referenced resource) within a decision shape in a BizTalk Server orchestration. Rules could be called in a decision shape or any other shape that allows a user to make a .NET call—for example, an expression shape; however they are primarily used through the Call Rules shape.
A policy can be tested before it is published and deployed. The Business Rules Composer provides a testing facility that allows instances of facts to be submitted to a policy and executed by the rule engine, with the results provided for analysis. The output identifies the sequential steps that take place: fact submission, condition evaluation, agenda update, and rule execution. When composed and tested, the policy is published and the published version is locked. In this way, the policy is locked down, and its behavior is well defined. A policy version can be used under a given set of circumstances, replaced by another version when those circumstances change, and reused if the original circumstances apply once again.
One of the most important features of the Rules Engine Deployment Wizard is the ability to update policies dynamically without interfering with the host application deploying the policy. An updated version of the policy can be created and deployed, and the host application can incorporate the updated version in near real time, without any change to the host application. This capability eliminates the significant overhead of redeveloping, testing, and redeploying an application.
The rule store is a repository for policies and vocabularies. By default, the rule store is a SQL Server database. Vocabularies and policies can also be maintained in a file system (a file store). Policies and vocabularies can be exported and imported between a SQL rule store and file stores.
The Run-Time Rule Engine processes policies, created by the Business Rules Composer, when invoked by a decision step within a BizTalk Server orchestration. The Run-Time Rule Engine is based on a forward-chaining inference algorithm that uses a three-stage policy execution methodology to resolve rule conflicts:
Matching. In this stage, facts introduced to the engine are matched against rule conditions. If a fact matches a condition of a rule, then that rule becomes a candidate for execution and is added to the rule engine agenda. An agenda is completed when all facts that are simultaneously present in working memory are matched to all active policy rules.
Conflict Resolution. The conflict resolution stage subsequently compares all candidate rules on an agenda to determine if any execution sequencing conflicts exist in the logic of the rules. If any are determined, the conflicts are resolved according to the execution sequence defined by the user. The engine then sorts the agenda based on priority (high to low). The rule with the highest priority will execute first.
Action. In the action stage any required actions are executed. Actions can assert new facts or retract/update facts into the rule engine, which causes the cycle to continue. This is known as forward chaining—where the match/resolve/act cycle repeats for rules affected by a changing fact base. The user has explicit control over when forward chaining should and shouldn’t occur (using the Assert/Retract/Update functions).
While every rule results in either a true or a false outcome, the ultimate outcome of a complex, chaining rule set can vary if the order in which the rules are executed changes. Furthermore, as noted in the preceding Action stage item, facts are very often derived from the execution of prior rules, and these facts are then asserted and evaluated against the conditions of other rules in a policy. Consequently, it is important that a rule engine be capable of resolving both explicit and implicit dependencies and contingencies in complex policies to prevent ambiguous results or unforeseen sequence steps, evaluations, and actions. To avoid irregular behavior, the BizTalk Server Rule Engine uses an “agenda” mechanism that matches received or generated facts against a given policy, in conjunction with a prioritization scheme, to resolve complex rule execution conflicts and ambiguities. The processing of a mortgage application that will typically contain multiple facts about the applicant (for example, income and loan amount) offers an example of how the agenda mechanism works.
Suppose the mortgage approval process requires the execution of two rules:
An income rule that determines that an applicant’s credit rating should be obtained only if the applicant’s income to mortgage amount ratio is greater than 0.2.
A credit rating rule that stipulates that an applicant should be approved only if the applicant’s credit rating is more than 700.
Note that the execution of the credit rating rule is dependent on the outcome of the income rule being a ratio greater than 0.2, which results in the action of a credit rating being obtained for the applicant.
When an application (a BizTalk Server orchestration) receives the mortgage application (an XML document), the rule engine determines the existence of the income and loan amount facts, but not the credit rating fact. The income rule will be added to the rule engine agenda because the facts needed to evaluate its condition are available from the loan application document. Because the only rule in the agenda is the income rule, the conflict resolution stage moves it on to the action stage where it is executed. If the execution of the income rule evaluates to a ratio greater than 0.2, then the resulting action is to obtain the credit rating of the applicant. Because the ratio value fact was not present in the loan application, the condition of the credit rating rule could not be matched to a fact; consequently, this rule was not added to the agenda.
The credit rating rule will not be invoked until the credit rating fact is supplied to the rule engine. Receipt of the credit rating fact triggers the rule engine to match the new fact against its rule cache. Finding a match, the credit rating is added to the agenda and moved to the action stage where it is executed upon, and the rule engine returns control of the process to the match phase—the match phase only reoccurs if the user declares a forward chaining scenario using the assert/retract/update functions.
In a situation where multiple facts are present, and they match rules that have a potential sequencing or action conflict, the conflict resolution stage resolves the sequence issue according to a prioritization scheme defined for the rules. Furthermore, the conflict resolution stage allows rules to be organized as policies.
The significance of this matching/conflict resolution/action methodology is that it allows the business rule developer to focus on defining the discrete logic for each individual rule, and not the execution contingencies or conflicts that may arise from the aggregation of the rules and their combined execution. Furthermore, rules can be modified, added, or deleted without concern for the organization or structure of the policy. This eliminates a substantial amount of development complexity that is inherent to writing procedural methods for business rule execution.
Supporting BizTalk Server Components
In addition to the primary modules and components that provide the core functionality of the Business Rule Framework, numerous supporting components and tools are included that facilitate the functional requirements and capabilities expected of an enterprise-class application:
Automated versioning and nonintrusive deployment of rule changes
Configuration of operational performance variables
Testing at different life cycle stages
Monitoring of rule processes and tracking of data from them
Security provisions to protect the integrity of business rules at every life cycle stage
Versatile capabilities for customization and integration
Following is a list of the supporting components and tools and a description of the features and functions they provide:
The Rules Language Converter. The Rules Language Converter is a plug-in component that simplifies the task of migrating policies from one rule language to another, or from one version of a language to another, by importing or exporting policies and vocabularies in an XML-based business rule schema language (BRL). Thus it provides a neutral and fully documented intermediate format that can be easily parsed and transformed by any XML editing tools.
The Rule Set Update Service. The Rule Set Update service is an autostart Windows service that is responsible for subscribing to policy deployment and undeployment events that occur when business policies change. In a typical enterprise scenario, business policies are deployed after being updated, and tested. Deployment consists of marking a rule in the rule store as “deployed” and optionally, publishing information about the updated policy to all interested parties. At this point, information about the policy is published instead of the policy itself. Running on a server that hosts a rule-based application, the Rule Set Update service then uses its polling mechanism to find newly deployed (or undeployed) policies and caches the information for subsequent use. The use of a publish/subscribe model for policy updates facilitates near real-time morphing of business policies without interruptions to applications using the policy.
The Fact Retriever. The Fact Retriever is a user-implemented interface, used to manage long-term facts used by business policies. The Business Rule Framework distinguishes between two types of facts: short-term and long-term. A short-term fact is one that will typically change with each execution instance of a rule, while a long-term fact remains constant for an arbitrary number of execution cycles. The only functional distinction between the two is in implementation: if a fact changes infrequently, rule processing efficiency can be obtained by saving it as a long-term fact and loading it into memory to reuse. By referencing this fact retriever in a policy, the user ensures that the engine (more accurately the policy class) will call the fact retriever to get long-term facts.
A policy can be configured to use a given fact retriever. At run time, the policy object will create an instance of the fact retriever, associate it with the rule engine instance, and update the set of long-term facts present in the working memory of the rule engine. It is the responsibility of the Fact Retriever to determine when the fact base has changed. A use can accomplish this in a number of ways: a publish/subscribe service to monitor long-term facts in the persistent store, a file update notification service, WMI events, or altering system messages.
The Rule Set Tracking Interceptor. The Rule Set Tracking Interceptor is a plug-in component that receives and processes output events from the rule engine. The rule set tracking interceptor can be used as a debugging tool or to track and monitor the execution of business policies either historically or in near real time. In a BizTalk Server production environment, you can track and monitor a rule-based orchestration using the BizTalk Server Health Activity and Tracking (HAT) module. The rule engine integrates with HAT to track and monitor activity relating to facts, condition evaluation, rule firing, and agendas.
Policy Testing Tools. The Business Rules Framework provides several options for testing policies at different stages of their life cycle:
Policy and PolicyTester objects can be used to programmatically test the policy. The objects evaluate sample facts against an XML policy file found in the file store. These objects can use the debug tracking interceptor mentioned below.
The Rule Set Tracking Interceptor can be used in debug mode to write tracking information to a file.
Pressing the “Test Policy” button in the Business Rules Composer to simulate the results of a policy execution when in the development stage.
Authorization Facilities. The Business Rule Framework allows an administrator to associate authorization groups to both policies and vocabularies. These groups can be further configured to have rights to perform different operations—read/execute, modify/delete, or both—on different policies and vocabularies. With artifact-level security on, when a new policy or vocabulary is created in the rule store, only the user who created it and the rule engine administrator have read/execute and modify/delete access.
While the Business Rules Framework is a new module of BizTalk Server 2004, it should be obvious from this overview that its functions and features meet the rigorous operational performance standards of an enterprise-class application. At the same time, the Business Rule Framework represents an innovative implementation of the Service Oriented Architecture paradigm. Every individual and aggregate level of functionality has been designed to be exposed, independent, and capable of being loosely coupled. Any policy component (vocabulary or rules) can be viewed, modified, or completely replaced at any time, without affecting any other process operation or running instance of the affected process. Furthermore, an in-memory object model representation of the policy is created, thus eliminating the need for any procedural implementation programming. The substantial development and life cycle efficiencies that this capability engenders alone are validation of the SOA paradigm around which the Business Rules Framework and BizTalk Server is designed.
While a short-term return on investment through cost savings is a reasonable expectation and incentive for adopting this technology, doing so also has significant strategic value that will yield long-term benefits. The next generation of Web services protocols will make it possible for business partners to engage in complex and context-specific business activities through the exchange of data and rule-based instructions. This will facilitate the automation of sophisticated business interactions that are presently beyond the scope of e-commerce technology. Trading partners will be able to use XML to encapsulate and exchange business rules representing contractual terms and conditions for pricing, discounts, cancellations, refunds, promotions, and so on. Each partner will be able to comprehend and execute these policies even if they are using different software applications. Business opportunities and requirements will be published publicly as rule-based business logic, and the ability to negotiate agreements (as well as execute and adjudicate those agreements) in an automated fashion will be one of the most important factors determining business success in the near future. This is already taking place through the facilities of the Web Services’ Universal Description, Discovery and Integration (UDDI) protocol and will be further advanced by a standardized XML representation of business rules.
Microsoft, more than any other information technology provider, has recognized and embraced the enormous possibilities of XML technologies. BizTalk Server 2004 is now in its third generation and the inclusion of the Business Rules Framework within it represents Microsoft’s commitment to propagating the use of XML and the SOA paradigm at the most fundamental business process level.