From Capabilities to Services: Moving from a Business Architecture to an IT Implementation
Ulrich Homann, Jon Tobey
Summary: Explains how to model a business architecture as a network of capabilities, and then transform the business architecture into a service-oriented architecture. (30 printed pages)
Principles of Service-Orientation
Relating Motion to Service-Orientation
Transforming Motion Artifacts into Services
Appendix A: Implementing Capabilities to Services Through BCF
Appendix B: Assessment Phase Examples
In our previous article we talked about the importance of using business architecture as the basis for a service-oriented architecture. In that article, we argued specifically that a business architecture modeled as a network of capabilities offers an architecture foundation that is ideally aligned with service-orientation. Both share critical attributes such as external, observable, and measurable behavior, with equal emphasis on required functionality and interaction. In this article, we will discuss how to develop a network of interconnected services from such a business architecture.
While there are many methodologies for determining business architecture in terms of business processes, there are not currently as many options for decomposing the business requirements into interconnected capabilities. In this article, we will use the Motion methodology as an example technique to get this interconnected architecture and transform the resulting artifacts into artifacts that are related to Web services technology. Of course, this is not the only way to model your business architecture, but it gives us a concrete example for our discussion.
Once you have defined your business architecture by extracting the structured network view of business capabilities and connections, and reconnected this view with the relevant business drivers, you are ready to transform the business architecture view into a technology architecture view—specifically, to a service-oriented view. This transformation will create a first approximation of the desired service-orientation that aligns business requirements and viewpoints with the technical implementation. This article focuses on concepts and strategies that will help your business decisions align with technical implementation strategies.
Every Motion project will be unique, and for each Motion project there is an infinite number of service-oriented approaches to support it. This means that there are many issues to consider along the path that this article traverses. For clarity, we include an outline of our approach:
- We will start with an overview of Motion: what it is, and what you get out of it.
- Then, we will briefly cover the principles of service-orientation, including the challenges that face an architect embarking on using service-orientation in a project.
- Next, we tie the two together, showing why Motion is an ideal method to capture business architecture, and relating it to IT architecture. This will explain not only why capabilities are important, but also why their connections are important and how Motion captures them. In the end, Motion will support both the business viewpoint and the architectural viewpoint, to make sure that the latter supports the business requirements.
- Once this viewpoint is established, we can lay out how you go from Motion artifacts to services. Here, we take the information in Motion, and begin to drill down into service design by exploring connections, clustering, and boundaries. Even with a well run Motion project, the designer has to make a lot of decisions, and we try to elaborate principles to guide you through this.
- "Appendix A: Implementing Capabilities to Services Through BCF" then takes you through a high-level example, using the Business Collaboration Framework to go from Motion artifacts to services.
- "Appendix B: Assessment Phase Examples" gives a drilldown of service-level expectations (SLEs) through various levels of Motion decomposition.
Motion is a methodology to first get an understanding of a business, and then to choose a particular project for improvement. Motion differs from methodologies with similar goals, in that Motion's focus is on understanding and defining what a business does (capabilities), rather than how it does them (business process management). The contention is that these capabilities are much longer-lasting than the processes that implement them, so that this model remains relatively stable over time.
Understanding a business as a set of capabilities has the additional advantage that such views translate well into services and service-oriented architectures. Therefore, the longevity of the business model also transfers to the IT implementation that supports it.
Motion is a four-phase process.
In the first phase, the team develops a capability (or module) map of the business, taking a generic module map (see Figure 1), and customizing it for their business by listing the most basic capabilities and decomposing them down to, at most, three levels (see Figure 2).
Figure 1. A high-level generic module map
Figure 2. Decomposing the module map
From this map, the team looks at the business value of individual capabilities, and uses various tests to pick an improvement project.
In Phase 1, the team picked the project. Phase 2 has two parallel paths. On the first path, the team turns to subject matter experts (SMEs) to collect data that the team needs for decision making. This includes process flows, organizational charts, financial data, and so on.
In parallel with this, the team starts looking at the project in detail. First, they "Go In." Here, they get a complete description of the project, by mapping capabilities at level 2 and deeper that are relevant to the project. In other words, they explore the module map relevant to the project at a deeper level than they explored the rest of the map.
Then, they "Go Up" and "Go Out." In these steps, the team sets context by looking at how the problem space relates to business performance, both internally (up) and externally (out). As you look up, how does this capability relate to other capabilities? For example, is there repetition among capabilities? As you look out, how does it relate to the business boundary? For example, can you push capabilities onto partners or customers, as airlines did with self-checkin?
"Go Down" is the conventional breaking down into component pieces that every methodology does, except the focus is on capabilities, not processes.
In Phase 2, the team got a firm understanding of the problem and how it related to other capabilities. In Phase 3, they assign capability connectors and attributes (business SLEs), and then build a stack of views, including:
Then, they relate the views of this stack to the map from Phase 1 to see where there are the dependencies, to make it clear what can and cannot be changed. If something doesn't contribute to the business performance, its process doesn't matter.
In Phase 3, the team understood not just the context of the project, but its detailed interaction with other capabilities. In Phase 4, they formalize discussion of what they can and cannot change, and if they change it, which changes have the greatest overall impact. This is where the team begins a structured dialogue about solutions, culminating in a recommendation to improve the project chosen in Phase 1.
Pat Helland wrote an excellent article about the basic principles of service-orientation. Those principles—among others—guide the transformation philosophy outlined in this article.
What Is Service Orientation?
Note This section is adapted from Design Rules, by Carliss Y. Baldwin and Kim B. Clark, 1999.
Simply put, service-orientation is a concept that naturally evolved out of the desire and longstanding efforts to modularize complex computer systems and the real world that those systems represent. Every year, more and more aspects of our lives become supported or automated by computer systems. As this trend continues, new and existing automation systems require more interconnections with other systems in order to reflect the interconnectedness of our real-world systems. The sheer breadth and relatedness of today's computer systems drives ever-increasing complexity. We use modularization as an organizing principle to make these systems more comprehensible and manageable. Service-orientation subsumes two major ideas of modularization:
- The first is the idea of interdependence within, and independence across, services. A service is a unit whose structural elements are strongly connected among themselves, and relatively weakly connected to elements in other units. Their structural integrity is such that each and every part of the service unit is required in order for the service to exist as a defined entity—taking away any part would disable the service. At the same time, service units may have important functional ties to other services, even though these links are tangential to the services themselves. The loss of such a link might mean that a service could not perform certain functions, but it would not completely prohibit providing service. For instance, an online travel service, such as Travelocity or Expedia, might offer currency conversion services for the convenience of international buyers. Losing this service might inconvenience some of their customers, but would in no way affect the core travel service. In other words, services are units in larger systems that are structurally independent of one another, but that work together to provide outputs and outcomes. The system as a whole must therefore provide a framework—an architecture—that allows for both independence of structure and integration of function.
- The second idea is captured by well-known terms in the distributed computing and object-oriented (OO) arena: abstraction, information-hiding, and interface. A complex system can be managed by dividing it into smaller pieces and looking at each one separately. When the complexity of one of the elements crosses a certain threshold, that complexity can be dealt with by defining a new abstraction layer, with a simple interface hiding the complexity behind the new layer. Business Protocols: Implementing A Customer Complaint System describes just such a layering. The user interfaces with a quality management interface, but he or she has no idea which of several services, based on service-level expectations (SLEs), actually implements the process. The abstraction hides the complexity of the element; the interface indicates how the element interacts with the larger system.
Discourse: Services Versus Components
There has been a natural evolution over the years of how software is constructed—first using functions, then objects and components, and now services. In the beginning, separating code into functions allowed software to be grouped into smaller and better-organized pieces that interacted with each other in deliberately designed architectural patterns. Then, objects provided the means for encapsulating per-instance data in re-entrant code. After that, components provided services for run-time activation. In all of these approaches, modularization depended on a coherent execution environment to ensure module interoperability.
Currently, services have taken center stage in software evolution. Services provide more independence between the pieces of code than functions and components did (see Figure 3), primarily because they do not require a shared execution environment to manage module interoperability—the necessary interoperability is managed purely on the communication or (network) wire level. They represent an example of "abstraction, information-hiding, and interface." However, because there is no "enforcer" in the environment, services must be their own enforcers: each service must obey the principles of service-orientation if they are to reap the benefits of service-orientation. Therefore, these modularization principles are both the price and reward of service-orientation. First, services always interact with each other through messages, and only through messages. They do not share run-time environments or other implicit assumptions about shared knowledge, such as configuration tables in common databases. Second, they are durable, in the sense they are not susceptible to failures of other services to which they are connected, and they are responsible for recovering from their own failure in a defined and predictable way.
Figure 3. Services versus components and functions
The business architecture developed through executing one or more Motion projects translates well into modularization. The resulting capability map represents the system as a whole, using the modeled business as the boundary for defining functional integration. The identified capabilities represent the units of structural cohesion (or affinity), whereas the connections represent the known functional interdependencies between them. The capability map shows what is weakly connected and what is strongly connected. Furthermore, capability attributes and connection details make it possible to start tackling the following issues (the level of precision depends on the level of details provided by the Motion deliverables):
- Interaction Inventory—What are all the interactions that services are expected to support?
- Interface Factorization—Given the interaction inventory, how are interactions grouped to start forming service interfaces?
- Service Factorization—Other than the interaction groupings, what are the factors that should influence the first cut of a services map? These include detailed sourcing information, a better understanding of service interface factorization (granularity), formal service-level agreements (SLAs) or informal service-level expectations (SLEs), issues of latency and frequency of interaction, sharing of data, and so on.
- Technical Capabilities—What are the current technical capabilities and constraints documented in the capability map? What opportunities exist to use technology to deliver the functionality required by business capabilities? The needs for specific technical capabilities (for example, database, authentication service, or integration service) start to emerge and define whether technical capabilities can be shared, and how they should be shared.
Going from Business to Technological Points of View
With a capability map that represents an architectural view of the business as a set of the business capabilities and their connections, we can see how these business services are interconnected. The outputs of one business service are necessary in order for another business service to perform its function. Such a model represents the fundamental business services or functions that are indispensable to the business, and their relationships to each other. Modifying the map by adding or deleting capabilities or connections between them represents a different business.
Note Capability boundaries do not necessarily equate to technical service boundaries: these are the actual capabilities or business services that a business supplies to customers, not services in the sense of service-orientation).
These indispensable capabilities and connections are annotated with properties that describe, in increasing detail, aspects that are less essential to the nature of the business. These aspects relate more to ephemeral qualities of business services, such as how the services are provided, the specific events related to their performance metrics, the people responsible for the service, and the systems that provide support for the service (see Figure 4). Some of these annotated properties relate the capability model to the business's financial and organizational models, and some locate business process flows and business events that drive performance metrics.
Figure 4. Association of capability to people, process/procedure, and technology layers
Because the capability map represents a fundamental view of the business as a collection of interconnected services, it can be related to another model of business services. That model is the technology architecture of the business's computer network systems. The minimum requirement for an agile technology architecture is to ensure visibility of those business services and key business service properties represented in the capability map in its logical architecture and real-world monitoring.
Business Services Are Built Around Interactions
In addition to determining the capabilities of a business, Motion focuses on the connections between these capabilities. Motion distinguishes three types of connections that are relevant to the business:
- Input/Output—Demonstrates the consumer–supplier interactions between capabilities. In other words, one capability is a supplier of business services to another capability that is a consumer of those services. This type of connection is characterized by a request from one capability to another, which provides a response to the requester.
- Supporting—Another type of connection is the supporting connection, where information is commonly passed from one capability to another. Exception handling of various kinds, inventory returns, and verification and inspection capabilities are examples of capabilities that would require supporting connections. Unlike input/output connections, supporting connections flow in one direction.
- Control—The third type of connection is the control connection, where one capability impacts the performance of other capabilities, not by supplying it with services, but by setting policy or performance requirements. The relationship is fundamentally a management relationship. Compliance, management policy, engineering, and regulatory business capabilities often have influencing relationships on other capabilities
You will use the capability inventory and the properties related to their interconnections, with the discipline of entity lifecycle analysis and supporting services analysis (see "Transforming Motion Artifacts into Services"), to arrive at a comprehensive interaction inventory. Then, determine which of these interactions are existing systems and what, if any, constraints are placed on these interactions by those systems. Initially, these are grouped by the business capabilities defined in your Motion project. Eventually, you will distill them into service operations.
Starting in the Right Place
Service-orientation by itself is interesting, but it is only when we base our architecture on business requirements that it becomes really important. The whole point, in fact, in going through a Motion project, and building a capability map, is to get this view on a business that we can correlate to our IT infrastructure. Once we align the IT and business requirements, we maximize both our architecture's flexibility and its longevity.
So, the biggest challenge to service-orientation is getting the right picture, which we dealt with through our Motion analysis. But moving from that picture to an actual implementation is not trivial: the two are aligned, but they do not have a one-to-one relationship.
Two of the principles of service-orientation are also two of its greatest implementation challenges—explicit boundaries and service autonomy. We must design services so that, except for defined message interactions, Service A is always independent of Service B. As long as we maintain the schema and contract, we are free to recode, redeploy, or completely replace them, independently of each other.
When we start to explore how business capabilities can be recast into a service-oriented system that starts to resemble a computerized implementation, we begin to realize that our business map contains quite a bit of "shorthand." The connection between the capability of Refine Existing Product/Service and Provide Service in the business map is a very complex one, and it summarizes a variety of formal and informal feedback mechanisms. The information that flows along this connection is pretty well defined in a general way, but for most businesses, it would be very difficult (and probably not useful) to attempt to schematize this connection into service messages. This is partly because the capabilities themselves are only partially automated, and they encompass complex service networks on their own.
Thus, before we begin to transform business capabilities into a more systematized and schematized services network architecture, we need to think more deeply about the context and characteristics of the capability network, especially in two distinct dimensions—the entity "lifecycle," and the supporting services required (How does a service know what it needs to know?).
- The "entity" lifecycle—Usually the work that is being performed by any given capability focuses on transforming information from one type to another—for example, from a Purchase Request to a Work Order, or from an Order to an Invoice. To ensure complete functionality from a business point of view, Motion already encourages business architects to validate the entity lifecycle as part of the business architecture design. In addition to providing the required list of business capabilities, a successful Motion project provides information to populate the relationship lifecycle. Table 1 shows a mapping from a business capability attribute to the appropriate lifecycle in the relationship.
Note Business capability attributes always focus exclusively on the externally visible aspects.
Table 1. From Motion to service-orientation
|Motion Artifact/Property||Service-oriented Artifact/Property||Design Considerations|
|Capability—Lower-level||Service—Set of related services: WSDL, WS-Discovery, WS-Addressing||First, catalog all interactions for a capability, and then regroup interactions into services. Capability boundaries don't determine service boundaries, but once you develop a full catalog of interaction, their affinities drive service boundaries.|
|Inputs and Outputs||Schematized messages (operation) or schematized private data||You need to determine what data is private to a service (not an interaction), by understanding all of the inputs and outputs for each interaction, and then seeing whether grouping interactions based on the shared input/output data works; if it does, the data is privatized.|
|Process flows (as in a capability relationship diagram)||BPEL4WS structure—WS-BusinessActivity rules|
|SLE properties||Operation—Port bindings (because the transport may impact SLE levels) WS-BusinessActivity (because the SLE may be related to a cluster of operations)|
|Capability operation frequency||This is important when designing the service implementation, because it sets a requirement for the service performance.|
|Capability variability||Operation—WS-Policy (if the variability is related to business policy settings)||This is important in considering the drivers of variability that might impact how the operations are defined (if variability is driven by input data), or how the service(s) that provide this capability are factored. For example, it may make sense to create multiple services, each with less variability. Mostly, this is an invitation to the designer to make sure that the sources and dimensions of the variability are well understood.|
|Agreement/contract-based capability, or one under regulatory control of some kind||WS-Policy and WS-Management might both play a part in capabilities that have this attribute|
|Functional complexity||As with variability, this invites the designer to make sure that what makes this functionally complex is well understood. May trade-off message schema and operations (having more operations with simpler schemas, or fewer operations with more complex schemas) and may factor into multiple services (operation clusters).|
|Complexity of interfaces of a capability|
|Rate of exceptions|
One simple abstraction of the activities in the entity lifecycle could be Create, Read, Update, Delete (CRUD) —in the sense that you would need to provide support for all of these activities, not in the sense of using this method of data handling.
- The required supporting services inventory—By and large, any work that gets done requires the availability of other, supporting business capabilities. Before a Purchase Request can be sent for approval, a catalog of available supplies has to be set up, configured, and provided to the employee who creates the Purchase Request. For instance, providing a catalog requires negotiation and sign-up of vendors, and a host of other dependencies. Of course, the design is recursive: each newly identified business capability requires examination through the entity, as well as the supporting services inventory.
By accounting for these dimensions of business capabilities, the service designer should be able to trace the business architecture and validate the business description. If any doubt remains, the service designer and the business architect need to reconcile and clarify the situation.
The key to service-oriented architectures is that the services exist in a complex network of relationships with each other. Very few services, if any, are likely to provide the kind of end-to-end processing in a single business area that characterizes today's applications, and none will contain any direct user interface support—services are "headless." Rather, to get the full cluster of functionality that we are used to in stand-alone applications, service-oriented "applications" will require many services. Initially, these are grouped by the business capabilities that we defined in our Motion project. Eventually, you will distill them into service-oriented operations.
The capabilities that each service provides, and the connections that they have with each other, determine the current and potential capabilities of the architecture. Current capabilities simply state what the service platform already provides. Potential capabilities describe what the service platform can deliver, given some additional service orchestration.
We combine the capability inventory, and the properties related to their interconnections, with the discipline of entity lifecycle analysis and supporting services analysis, to arrive at a comprehensive interaction inventory. Then, determine which of these interactions the existing systems will provide and what, if any, constraints are placed on these interactions by those systems.
As we define these interactions, we identify several specific properties that they need to have. These include:
- Name (to uniquely identify it).
- The initiating capability or "actor" (requester/source).
- The target capability or "actor" (server/destination).
- The data that the requestor provides to the server to perform the interaction.
- The data that the requestor cannot provide to the server to perform the interaction.
- All possible outcomes of the interaction, both successful and unsuccessful.
- The data that might be returned to the requester as part of an outcome.
- The data that might be accumulated or reflect the change in state resulting from the interaction outcome.
Interactions Do Not Translate One-to-One with Services
Some of these interactions are directly related to the connections identified between capabilities, but even these may not be one-to-one relationships. Some individual interactions that are clearly understood on a business level require multiple schematized interactions as you describe the system-to-system interactions. A purchase at the point-of-sale generates a payment between the purchaser and the retailer, but the variety of payment mechanisms (cash, check, credit card, and debit card), as well as the details of the purchase transaction options (simple purchase, purchase with cash back, purchase with manufacturer discount, and so on), all drive further complexity into the schematized interaction. A designer trades interaction complexity for simpler interactions, but more of them.
Also, from the perspective of the business, the individual interactions developed in this analysis are not relevant in themselves. They are relevant only insofar as they support much larger transactions, operations, and activities. For this reason, we need to consider how to surface these broader activities in our services model, and ensure that the forest is not lost among the trees. There are some standards that are helpful in this area, including WS-BusinessActivity (and the WS-Coordination that is its foundation) and BPEL4WS. As with most of the WS standards, these are especially helpful in illuminating the concerns and issues that need to be addressed in this space, even if they don't provide the full solution.
Given this perspective, it is important to keep firmly in mind that in a service-oriented architecture, the power and agility of the architecture lies in the connections between services, and in the fact there is a low level of friction associated with making these connections. This means that the normal situation for any given service is to be enmeshed in a complex set of relationships with other services, as well as with the network as a whole. When designing a service-orientated architecture, the context of these relationships within the network is as important to the design and factoring of the services as are the functions that the services provide. These relationships are rather complex, and they generally follow a lifecycle that can be summarized as follows:
- Establish a relationship.
- Conduct "business" based on the relationship.
- Assess the health/value of the relationship.
- Inevitably, change the relationship.
A completed capability map will give you all the information to do these things with the available technology.
Establishing a Relationship
This includes all of the activities that lead to the successful setup of a relationship. There are a number of aspects to consider: configuration information, common settings, reference data, and policy (corporate, regulatory, compliance expectations raised by business partners—for example ISO9000:2000-compliance, and so on). Today, many application programmers share this kind of information implicitly, through parameters stored in databases or shared configuration files. In a services environment, implicit sharing of information violates the principles of service interaction through messages only. So how is this kind of information made available to services that need to connect with each other? Although it is not possible to cover all possible types of information required to establish a relationship, there are two critical types of information to consider thoughtfully:
- Reference data—Reference data is information published across service boundaries. For each collection of reference data, there is one service that creates it, publishes it, and periodically sends it to other services. Reference data varies, from information that is specific to a given business—for example catalogs, address books, and customer lists—to information that is generally considered configuration information required to synchronize independent services—for example, country, monetary symbols, and so on.
- Policy—Policy, in the sense of establishing the relationship, focuses on the variability or options available for services to conduct their business. Policy ranges from topics such as the method of depreciating fixed assets, to scheduling rules (How often and when does reference data expire and need to be refreshed?) and rules for the subscription of reference data (Who is eligible? When? How frequently does synchronization occur? and so on).
There is no general rule-of-thumb with respect to using policy sensibly. However, one should think about all aspects of a relationship needed for fully functional interactions as good candidates for policy assertions. The aspects should cover all requirements and expectations for a successful relationship.
Conducting the Relationship
This includes the activities that focus on the work that the service has agreed to perform. This area is the one covered most completely by the existing and emerging Web services standards.
This involves understanding the delivery quality, timeliness, and so on, that the service agrees to provide, and making this available and visible to other services and the network managers formalized as service level expectations (SLEs). In general, this type of information is not available through indirect measures, such as message delivery, or other technical information, but rather focuses on the execution of the specific work—for instance, payment for employee XYZ for the current pay period.
Generally, this kind of assessment has been associated with business metrics or key performance indicators (KPIs), but it is not necessarily restricted to those kinds of measurements. For example, errors per pay period, measured in instances of errors, or monthly forecasts, measured in U.S. dollars, are common KPIs. "Appendix B: Assessment Phase Examples" contains detailed examples.
Because of the strict standards for service interaction using messages, any unforeseen change to the relationship is likely to impact these messages and thus require a completely new service interface. This kind of change is relatively traumatic for an established system, and it can be avoided by explicitly sharing variability information about what the services can currently accept and process. In other words, we have to synchronize certain assumptions in order to interact. An example would be changing your accounting service from a cash-based accounting to an accounts receivable system. You could design an interface that would send the right information regardless of the service. Thinking through these kinds of variations in service behavior, especially regarding how the service might behave in different configurations and deployment environments, reduces the requirement to create new services to accommodate such foreseeable variations.
In many cases, the variability options are exposed as part of the Establish phase, or as part of the assessment opportunities or abilities. These variability options may be part of the data content of the normal interaction messages between services, but they may also be controlled by private data in a service (data internally maintained and never directly exposed outside of the service), which is updated based on a policy change operation on the service. As such, the change may be transparent to related services, but have an implicit impact on the behavior of the changed service. Rarely should you change the conduct behavior or work that is done, because this generally alters the relationship to a point where it represents a new service, and thus a new relationship. New relationships should not be addressed using change behavior, but using a separate relationship that covers the complete relationship lifecycle.
Thus, we need to consider how services will expose this behavior effectively. Web services standards describe pretty well, through Web Services Description Language (WSDL), how the service will interact with other services (or any external actor) once they are pointed at one another, but they are not as effective in classifying interactions or dealing with change. How does a service assess that relationship, which is a crucial aspect of the health of the service network? Finally, how does the service advertise changes (or variations?) in its relationships? (See "Razorbills: What and How of Service Consumption.")
Interactions Need SLEs
Business service-level expectation (SLE) information collected during Motion is usually crucial, not simply to the business and services architectures being developed, but also to the success of the project itself, and to the system implementation being planned. It is tempting, therefore, to treat business SLE information as simply a basic requirement for the implementation, rather than as artifacts themselves. Business SLE properties are artifacts that almost certainly we want to be highly visible in the implemented system. This requires interactions among services to define, capture, and process business SLE information. Similar to compliance and management types of capabilities, SLE information represents a cross-cutting concern for service-orientation, but one that is tied to the relevant business capabilities. These are not necessarily identical with key performance indicators of the business, although they are likely to be closely related. Rather, they are tied to the definition and understanding of the business capabilities. As the business architecture needs to be visible in the implemented system architecture, these business SLEs provide the basic measure of the health and well-being of the business as framed within the business architecture. Business SLEs can be approached in a way similar to policy, but WS-Management might also provide some direction in how to approach SLE definition and monitoring.
All of this analysis will result in a catalog of interactions and the interaction properties described previously. You can add some of the additional properties of capabilities and their interconnections that you captured in your Motion project to this catalog as annotations that will assist you when making some boundary decisions and setting implementation constraints on the services design. For example, the fact that a capability requires human interaction almost certainly means that some of the interactions associated with that capability will need response times that are acceptable to humans. The number of humans that interact with the capability, and the frequency of the interactions, are also important in the design of the code that will implement these services.
At the root of the problem of defining robust and relevant service-oriented environments is the need to properly define service boundaries—that is, identify the functionality that makes sense to expose externally as a black box. It is of utmost importance that the boundaries be designed with the greatest possible consideration, because the definition of the boundary dictates service SLEs (which are not the same as business SLEs; see Maarten Mullender's Weblog) and will greatly influence the flexibility and durability of the overall architecture. A common mistake that developers make is to surface the internal data structure (schema) to the outside world, simply because the various programming environments supporting service-orientation allow it. Watch this closely, especially in the area of data association and composition relationships. For example, Microsoft .NET provides easy access to class serialization and deserialization capabilities. Instead, architects and developers should focus on the verbs of the system (capabilities) rather than the data stored privately.
In Motion, you experimented with clustering, refactoring, and decomposing capabilities based on business influences. Clustering capabilities means changing the standard interactions of a capability with another capability, thus influencing decisions about implementation, deployment, management, and so on. Refactoring involves decomposing capabilities, and then grouping the capabilities in new ways. Decomposing capabilities requires splitting them into more basic capabilities. The clusters of capabilities and their connections that emerged after completing this step are a great foundation for a service-oriented design. If you have not completed this step in your business analysis, you need to do so, because the transformation of the business architecture into a robust service-oriented architecture requires clusters of connected business capabilities that truly represent the requirements of the business.
Although the clusters of business capabilities and connections represent a great first approximation for a service-oriented design, you must evaluate technical influences before implementation. You need to analyze the network of connected business capabilities through various lenses of affinity. Affinity points to factors that drive close communities or grouping. Since we are analyzing a network of connected business capabilities in order to design a network of interrelated services, identifying and understanding the factors that determine grouping is critical.
Again, the clusters that emerged during the business analysis phase are a solid base, and they should not be disregarded during this design phase. Below are some well-known factors that drive affinity in communication-centered architectures:
- Affinity of Critical or Common Private Data—If the to-be-implemented business capabilities share a high degree of private data that is commonly manipulated, and if the manipulations have immediate (real-time) effects (that is, if coherency is high), it is a good indicator for a close grouping. As an example, consider the following business capabilities: accounts receivables (AR), accounts payables (AP) and general ledger (GL). Closer examination of the private data required in order to perform the activities that define these capabilities shows that about 90% of the private data is common to all three capabilities. Furthermore, update activity in the GL capability requires immediate update of critical private data in AR and AP. Either factor indicates a services boundary combining AR, AP, and GL into one service. The fact that both factors are present is almost a sure sign that splitting (creating separate services out of AR, AP, or GL) is not a good design decision in this case, and that it should be considered only if strong business or technical benefits can be presented. The same is true in the reverse, if either the degree of sharing and/or the coherency need for private data is low, splitting into separate services is, at the very least, safe.
Figure 5. View (partial) of the financial management capabilities and their connections in a typical company
One type of private data is a legacy application or system data. If the capability is currently implemented by an application that does not conform to service-orientation principles, but the new system architecture is service-oriented, a common approach is to "wrap" the legacy application with a service. The same principles cited for data apply here. Even without a detailed knowledge of the legacy application, it is best to approach it as if it were itself a separate service. Maintain structural independence from the application and its private data as much as possible. Interact with it through messages or separate executable packages, rather than from within its execution environment, if possible. This will result in a service that is durable, manageable, and a good citizen in the service-oriented universe.Note Other considerations—such as deployment—will also factor heavily into boundary decisions, as outlined in more detail below.
- Affinity of the interactions—Similar to the affinity train of thought exposed in the preceding point, the affinity of interactions represented by the connections between the business capabilities presents strong indicators for boundary decisions. This analysis focuses on determining the coherency requirements of interactions, similar to the analysis of private data.
- Density of interaction—Density of interaction follows very similar arguments as affinity of information. If the interaction patterns are very dense (frequent), similar to function calls or object method invocations, they generally indicate a service boundary. In other words, capabilities that show dense interaction belong to the same service. If you must decompose and rearrange, it will be necessary to think about removing connections, as demonstrated in the ABC Corp. example.
- Time constraints on the interactions—Time constraints deal with latency issues—the time it takes for one round-trip interaction between two capabilities to be completed. It makes sense to draw service boundaries around clusters of capabilities that have similar latency requirements for their interactions, either high or low. Service levels will play a key role in determining latency constraints or opportunities/flexibility. "Appendix B: Assessment Phase Examples" provides several examples of service levels and the associated indicators.
- Transactions or referential integrity of update activity—Pat Helland eloquently outlines the current debate about transactions, and argues that, due to the nature of service-orientation, it is not advisable to support or require atomic consistent isolated durable (ACID) transactions between two services. ACID characteristics between two capabilities in the case of update activity (in this sense, creation and deletion activities are also part of update activity) is a sure sign that a service boundary should be drawn around those capabilities.
- Real-world separation of work activities—Physical process breaks in the real word is another clue as to where to place service boundaries. For instance, Place Order and Ship Order would be naturally separated by the physical process, and therefore they would be in separate services.
- Designer determination/choice—Experience and designer insight are great sources for solid design. While not very scientific or easily performed, it is a completely valid technique and, in a lot of cases, the source of innovation. However, the technical designer performing this step needs to tread lightly and to thoroughly understand the decisions that drove the clustering choices of the business architect(s). Modifying business-focused clusters might jeopardize the alignment with the business drivers, and lead to a less useful and less stable architecture.
Understanding Operations and Interactions
As mentioned previously, determining service boundaries is critical for service-orientation. Affinity already provides powerful insight into the factors that ought to influence service boundary design decisions. However, there are other factors that deserve consideration, in addition those outlined during the affinity discussion. Note, however, that affinity needs to be well understood before any other considerations.
One important outcome of autonomy is the ability to move services to different locations or, more precisely, to source them by different providers, either internally or externally. To prevent impairing this critical ability through uninformed boundary decisions, the designer needs to differentiate service boundaries by at least two additional criteria:
- Operational requirements—Providing a service to a specified service level requires careful consideration of operations. Although this article will not delve into details of services operations, operational considerations are important for a number of reasons. Chief among them is that operational boundaries determine sourcing boundaries. They are the set of connected capabilities that need to be managed cohesively in order to provide the expected level of service. Compliance and governance—such as Sarbanes-Oxley, Basel II, HIPAA, or internal compliance requirements that are individual to any given company—are further influence factors for operationally influenced boundaries.
- Interaction requirements—Although operational requirements might draw a boundary around one set of capabilities, you will never see this boundary as an interface: the work provided by the capabilities requires yet another boundary or set of boundaries. For instance, a service-oriented redesign of an enterprise resource planning (ERP) application might recognize the business capabilities Accounts Receivable, Accounts Payable, and General Ledger (AR, AP, and GL) as part of the financial management part of the solution environment, and a service boundary might be drawn encapsulating all three capabilities into a single service.
Although this is certainly a solid conclusion, upon closer examination, the relationships within the financial services capability group reveal powerful semantic connections among themselves, whereas the relationships to elements in other units are relatively weak. For example, the GL capability interactions are simply requests to debit or credit a specific account.
This indicates that there are two service boundaries to be drawn:
- An operational service design focused on implementation details such as common regulations, shared code, databases, and so on, in order to deliver upon the determined service level.
- An interaction service design focused on the implementation of the connections revealed during the business capability design.
The operational design would result in a financial management service containing the implementation of the AR, AP, and GL business capabilities. The interaction design would yield three service interfaces: AR, AP, and GL. That is, users would still interface with the individual services, not with the aggregate service. Any change in sourcing would have to happen on the operational level, with the requirement that the interaction services be provided by the new provider of the operational service.
Other Considerations to Determine Useful Boundaries
Understanding affinity and effects of operational considerations, as well as required interactions, provides a powerful framework to help the service designer approach the daunting task of determining robust and useful service boundaries. Furthermore, these two basic sets of considerations are closely aligned with the results of business capability mapping, resulting in a business architecture expressed in structured networks of connected business capabilities. However, depending on the level of capability mapping you have accomplished, you might also need to consider other strategies to determine service boundaries.
So, here again, we see both the autonomy and interdependence of services. This way, the reality of variations can be modeled in a way that fits with the chosen business architecture, and in a way that further helps drive sound service boundaries. For instance "collect payment" may be a check, debit card, credit card, online transfer, or cash. The interface should remain the same, regardless of the present payment method. We've covered this in a series of articles, published on MSDN, that focused on an approach labeled Business Protocols, and that outlined the theory behind these ideas and a practical implementation using Microsoft BizTalk Server 2004.
The purpose of Motion is to describe a particular business architecture in terms (capabilities and connectors) that allow translation between business requirements and IT requirements. From the capability map and the capability inventory forms, you have a catalog of the business services that the business needs. To a large extent, you understand what systems, if any, provide those services today, as well as the limitations and strengths of those systems, and their relationships with each other. With this description in hand, an architect can move from Motion artifacts to service-oriented artifacts.
The benefits of service-orientation are well known. How to model a business so that its service representation effectively supports a business, however, is another matter entirely. Motion gives you a tool to model the business architecture. This article has tried to guide you through the through the process of moving from the Motion artifacts to the service-oriented artifacts. Modeling an entire business and converting it to service-orientation would be a Sisyphean task. By design, Motion does not model an entire business, but only some aspect of the business that needs improvement. This way, the scope of the modeling, and the resulting services, should be implementable and not overwhelming. Each project can build on the success and structure of proceeding projects, so that in real-world applications, businesses will tend toward service-orientation, rather than encompass it in a singular effort that is doomed to fail. However, this requires that we build service-oriented architectures with the principle of expansion in mind.
After completing a Motion project, you have a business viewpoint that uses business terms and vocabulary for your business's requirements, processes, and interactions. We now need a way to transform these to terms closer to the implementation layer. Because the business capability layer is abstract, and not necessarily as complete or semantically rich as required for software engineering purposes, a method that allows an almost "automatic" transformation is desirable. Note, however, that no framework will truly automate the business requirements expressed as capabilities into fully functional software implementations. The aim, therefore, is to identify patterns of interaction—involved roles and functionality that can be mapped or associated with established best practices and well known patterns. One such set of business collaboration patterns and practices is the Business Collaboration Framework (BCF).
BCF allows you to translate from a business viewpoint to an information technology viewpoint, preserving the relevant information. It does this through pattern recognition and a meta model that defines the syntax and semantics for each view.
Starting with interactions defined in the Motion project, you can deconstruct them until each activity represents a reciprocal transaction between two capabilities. These are called collaborations. Collaboration is a set of transactions conducted by two or more capabilities in order to achieve a certain goal or objective (see Figure A1). Objectives are elaborated in the form of requirements and constraints providing a set of rules for the collaboration.
Figure A1. Collaborations connect capabilities
Collaboration provides modularity to transactions (see Figure A2).
Figure A2. Collaboration is a reciprocal business transaction between two capabilities
The following are examples of collaboration patterns:
- Escalating Commitments
Once you have identified the collaboration pattern, you can determine the transactions that are necessary to support the collaboration. A business transaction is a set of business information, and an exchange of messages (the business information) between two capabilities that must occur in an agreed-on message format, in an agreed-on sequence, and under certain time-period conditions. They have state, and are structurally built by exactly two "opposing" actions: a requesting activity, and a responding activity. These transactions abide by certain conditions that apply to the correct processing of transaction patterns. Once we get to this level, we can precisely define who does what (roles), relationships to the other capability (trust), commitments, expectations, ownership, consumption and production of resources, and so on. To those with software programming experience, the conditions may resemble a domain-specific programming language (DSL) in the domain of "transaction semantics." (See "A Software Factory Approach To HL7 Version 3 Solutions.")
In going from a process to collaborations, we can strictly define the transactions in the collaboration into a transaction pattern, which is defined as "an immutable set of characteristics and properties." There are six recognized business transaction patterns:
- Commercial Transaction—Models the "offer and acceptance" business transaction process that results in a residual obligation between both roles (a party is represented by a capability, and not necessarily by an explicitly identifiable business partner) to fulfill the terms of the contract.
- Query/Response—Queries for information that a responding role already has—for example, against a fixed information set that resides in a partner domain.
- Request/Response—An initiating role requests information that a responding role already has. Used when the request for information requires a complex interdependent set of results.
- Request/Confirm—An initiating role requests confirmation about its status or state with respect to previously established contracts, or with respect to a responding role's business rules.
- Information Distribution—Effects an informal information exchange transaction that therefore has no non-repudiation requirements.
- Notification—Effects a formal information exchange transaction that therefore has non-repudiation requirements.
Together, these six fundamental transaction patterns cover all known legally binding transactions between two decision-making applications, as defined by the ISO Open-EDI (ISO 14662) Information Standard, which is the original basis of all EDI standards, such as EDIFACT, as well as the ebXML idea.
The specific transaction patterns used in a collaboration are based on extracting information from business domain experts according to a standard script, shown in Figure A3.
Figure A3. Determining the appropriate transaction patterns
Recognizing these patterns then allows the definition and construction of predefined templates and solutions.
Service Interaction Patterns
A service and its agents are the facade that exposes the transactions.
Service interaction patterns are stateless message exchange patterns that are interchanged as messages between two services in order to construct a dialogue. There are two types of messages: action messages (sometimes called documents), which have business information, and service messages, which report on processing the action message processing.
All action messages, as they are being consumed by a service, need to go through a technical evaluation process to verify whether the message is well-formed. This process is based on a document-processing framework of the following steps (see also Figure A4):
- Grammar validation—Verifies that the syntax grammar of an action message is valid (usually only the header of the message is checked at this step).
- Sequence validation—Verifies that the collaboration control information is valid with respect to the transaction specification.
- Schema validation—Verifies that the message schema is valid with respect to a message guideline agreed to by both partners. It is recommended that message receipt be acknowledged after this validation step, to ensure that documents are "readable" as well as "accessible."
- Content validation—Verifies that the content of a message is valid with respect to any business rules that govern the formation of a contract. It is recommended that business acceptance be acknowledged after this validation step.
- Activity processing—Processes the transaction request in the initiating business action message.
Figure A4. Validation through business interaction patterns
Transforming Service Interactions to Message Exchange Patterns
In the "Service-Oriented Integration" chapter of the Microsoft Patterns & Practices book Integration Patterns, the authors refer to four common message exchange patterns, summarized in Table A1.
Table A1. Message exchange patterns
|Message Exchange Pattern Name||Description|
|Request-response||The endpoint receives a message, and then sends a correlated message.|
|One-way||The endpoint receives a message.|
|Solicit-response||The endpoint sends a message, and then receives a correlated message.|
|Notification||The endpoint sends a message.|
Although it is not feasible to associate the business connectors outlined above with a specific message exchange pattern, it is important to analyze the business requirements expressed in the business connector, and to choose the least intrusive or complex exchange pattern for the message interaction. Patterns that require correlation are more expensive in terms of computational requirements and synchronization of interaction dependencies—that is, both services have to identify data (for example, Purchase Order IDs or employee IDs) of the message topic, as well as the message sequence. Patterns that are one-way generally do not require that level of synchronization.
The exchange of a series of one or more transactions, within a collaboration, form a legally binding contract (see Figure A5). Typically, a business contract is used to legally bind parties to a clearly stated intention (promise or obligation), and to define the responsibilities of each party.
Figure A5. Contract components
When talking about transactions, there is a common misconception that they can only be between enterprises; but in fact, a lot of collaboration exists within the enterprise, and these are also covered by service contracts, and they also can be modeled exactly as above. The primary difference is the degree of trust: more business trust means fewer constraints and fewer things to manage for. We will base our example around such a collaboration example, because of the following considerations:
- Within an enterprise, it's easier to identify and model service providers and service consumers.
- Within an enterprise, it's easier to create non-negotiable service contracts where the service provider defines service behavior.
We will start from a retail company's balance sheet, with the following basic business equation:
Working Capital = current assets – liabilities
Current company assets can include customer orders, and company liabilities can include distribution costs, plus the costs to purchase product. So, as soon as a customer orders a product, we would like manufacturing to report this to accounting, changing the liability to an asset. This removes liabilities being tied up on the company books for new product purchases; or, the retailer could use those assets (investment money) for something else.
From this example of a business process, there are a few key interaction requirements that stand out, and that are critical for successfully operating the business equation to its potential:
From Figure A3, we see that the Notification Transaction Pattern can tell us exactly when the customer places the order. The notification pattern specifies the exchange of a notifying business action message, and the return of an acknowledgment of receipt business signal. This pattern is used to model a formal information exchange business transaction that therefore has non-repudiation requirements.
Taking a closer look at Case #1, it is essentially the Commercial Transaction Pattern.
From a design point of view, now that we know how the transaction behaves, based on what events should be occurring from a business perspective—the standard Service Interaction Patterns show how to arrange the actual exchange of messages back and forth between the two roles, recognizing that there might be other technical intermediary roles involved in the relaying of the messages between the two principal business roles (such as a third-party Web service doing data translation).
To keep this example system, let's assume that there is no intermediary service, and that there's a direct relationship between the two business principals. In this case, the manufacturing department's requirements for the requesting business activity property values for the notification transaction pattern are as follows:
Recurrence = 3
Non-repudiation of receipt = TRUE
Non-repudiation of origin & content = TRUE
Authorization required = TRUE
TimeToPerform = 24 hours
TimetoAcknowledge = 2 hours
TimetoAccept receipt = 1 hour
The accounting department's requirements for the responding business activity property values for the commercial transaction pattern are as follows:
Non-repudiation of origin & content = TRUE
Authorization required = TRUE
TimeToPerform = 24 hours
TimetoAcknowledge = 2 hours
TimetoAccept receipt = 1 hour
At this point, we have the all the business attributes necessary to form a "Service Definition," and we would need to start mapping these parameters into the patterns, and the patterns onto some Web services profile(s).
Table B1. Assessment phase examples
|Motion Service-Level Attributes||Example 1||Example 2||Example 3|
|References the SLEType entity (foreign key with a specific enumeration reserved for later tool releases) and identifies a specific way to measure a service level.||1|
[Quantity per Duration]
|Uniquely names the service-level definition.||Errors per pay period||Monthly revenue forecast||Production target for location|
|References the capability to which this service level applies. Examples refer to the generic capability map that is available as part of the Motion methodology||22.214.171.124.1.|
[Forecast Customer Sales]
|Names the unit of measure for the service level. For "Duration" type service levels, this should be a time period. For a "Monetary Cost" SLE type, "Dollars" or "Thousands of dollars" would be appropriate.||Pay period||Dollars||Raw number|
|If the SLE type references a "Throughput" type of SLE, this field indicates the length of the measurement period for throughput.||Twice per calendar month||Weekly||Per day, per location|
|Specifies an actual (current status/performance or historical performance) measurement of the SLE, such as the number of days of duration, the number of items completed for throughput, the amount of dollars for monetary cost, and so on.||12||$7,031,250||5,833|
|Specifies a target for future performance, such as the number of days of duration, the number of items completed for throughput, the amount of dollars for monetary cost, and so on.||0||$7,804,688||6,000|
|Specifies how much variation in performance (that is, from a goal) is tolerated before a variation is noted or a notification is sent. For example, when a variance threshold is exceeded, an e-mail message can be sent to appropriate management personnel.||>5% increase from prior period)||Null||Null|
|Provides a detailed description of the SLE for this capability.||In this case, the quantity in the SLE is an error rate per pay period.
Pay Employees is a key operational SLE for the business. Although paying employees is not in any way strategic to the business, because it has such a major impact on the morale of the employees of the business, it is very important that we know when there are defects in this particular function.
|This SLE is sales revenue per time period, and in this case, it is a roll-up of similar SLEs by region, and by sales person at the more detailed level.
Forecast Customer Sales, like Ship Product, is also a key performance indicator for the business. However, because this function is commonly performed at the management level, automated notification of variance is not necessary.
|This SLE is how many units of product are produced in a given manufacturing facility per day. This SLE will be correlated with other SLEs, such as quantity of product shipped, and error rates of products fabricated, to gain insight into best practices opportunities (from error rates), inventory-level management (by comparing shipped with produced), sales order throughput time (from order to production), procurement management (trending of production can trigger the need to buy more supplies). Threshold variations will result not from raw production numbers, but from correlation with orders, and that is the reason why the threshold information is null here.|
About the author
As a solutions architect, Ulrich Homann is responsible for the design and architecture of Web services and integration of the .NET platform with providers and consumers of business applications. Previously, Uli was program manager in the Enterprise Program Management team and responsible for the development relationship with key partners, mainly SAP AG. Uli also served as part of Microsoft Consulting Services, where he was responsible for several key distributed database application projects in Germany. Prior to joining Microsoft in 1991, Uli worked for several small consulting companies, where he designed and developed distributed systems.