What Are Composite Applications?
Globalization, specialization, and outsourcing require people to work in more collaborative ways than ever. This trend requires a matching change in the tools that information workers use to gain insight, collaborate, make decisions, and take action. Today, most business applications are effective at automating transactions, but they do not enable rich collaboration across functional boundaries. This usually leads information workers to use personal productivity tools to perform the complex interactions required to conduct business. However, this in turn leads to a loss in productivity, as users are forced to cross from one set of tools to another, often manually transporting the data through means such as cut-and-paste operations. These gaps between different business applications and productivity tools must be bridged for information workers in a way that is seamless, synchronized, and secure.
Ideally, it would be easy to stitch together different business applications and enable collaboration for information workers—but in a way that complex interactions between people can plug easily into structured business processes. Unfortunately, however, this has not been easy to do, as today's business applications are usually monolithic and difficult to modularize. There has been a lot of discussion in the industry around composite applications that are quick to build and deploy, and easy to change when the business changes. The stated goal of these kinds of business solutions is to make business processes more efficient in the real world, as opposed to automating idealized views of process that no longer meet the needs of the business.
This is certainly a laudable goal, and there have been a number of industry efforts in this direction. However, there really has not yet been an effective way to enable composition in a way that is contextual, collaborative, easy-to-use, role-based, and configurable. In many cases, the platform, tools, and architecture for composition introduce new technical complexity that is overwhelming in itself, and organizations' skill sets are not up to this challenge. This white paper will introduce the notion of composite applications as an architectural pattern for solving this problem. Subsequent white papers will show how the 2007 Microsoft Office System provides a good platform for building such composite applications—called Office Business Applications (OBA)—and how to pull in other platform technologies needed for building these kinds of applications.
Composition refers to a way of delivering enterprise solutions by assembling them from prebuilt components, instead of building them from scratch. It also includes personalization and customization abilities, so that users can easily and quickly modify specific functionality in the solution. This book will also show that moving to a composition architecture requires a fundamental shift in thinking, both for solution providers and end users of enterprise solutions. This is analogous to getting a prefabricated house assembled for you, instead of having it custom-built.
Figure 1. Building an enterprise solution using self-service architecture is like assembling a prefabricated house.
However the benefits are substantial, as composition provides the means to achieve agility, adaptability, and alignment in the enterprise.
Let us look at a typical business process, as shown in Figure 2. This is the case in which a sales person has a contact lead and wants to convert that into an actual order. If you look at a typical business application for sales automation, a lot of work will have gone into creating a perfect business process for this: create the lead, decide when it is qualified, create the opportunity, perform a quote, close the quote, complete the sale, and finally invoice the customer. It is this "perfect" process that is shown in Figure 2.
Figure 2. A perfect sales-automation process (Click on the picture for a larger image)
However, in reality, the process really looks like Figure 3. To create the opportunity, someone has to obtain specs from the customer to understand what the customer really wants. They have to decide if this opportunity is valid, and if it is something that can be feasibly delivered. They must cost the solution (probably using Excel), and this goes on. Clearly, the majority of real-world processes are not typically part of an ERP system or a CRM system, but these steps are integral to the process and just as important.
Figure 3. The sales-automation process in the real world (Click on the picture for a larger image)
We call this failure of traditional business applications to capture the real-world processes and the work that people do the results gap. It goes beyond processes, and includes problems like finding and accessing the correct data to make the right decisions. Ultimately, it means that IT investments frequently under-deliver on the promises their proponents make.
The results gap has some critical consequences for the enterprise. This is called out in a Gartner, Inc., research note entitled "The Knowledge Worker Investment Paradox," by Regina Casonato and Kathy Harris (July 17, 2002), which states:
In most enterprises, an employee will get 50 percent to 75 percent of their relevant information directly from other people.
More than 80 percent of the enterprise's digitized resources are not accessible to the enterprise as a whole, because they reside in individual hard drives and in personal files.
The individual owns the key resource of the knowledge economy—tacit and explicit knowledge—and most of that knowledge is lost when the individual decides to leave the enterprise.
The space between traditional business applications and the productivity tools that information workers use on a daily basis is very large, so there are a number of business processes that are affected by the results gap. For example, business processes like sales and operations planning (S&OP), collaborative product design, new product introduction, and so on often lack out-of-the-box solutions. While enterprises look to vendors to provide applications for these business processes, it is often hard to find packaged applications that fit the business need exactly. That is because these processes are very collaborative, instead of strictly transactional, and the solution space is characterized by semistructured data and processes.
Often, when packaged applications are unavailable, custom solutions are built. In most cases, business applications are built on top of an applications platform, instead of built from scratch. The choice of platform has different implications for the IT department that oversees development of solutions, as well as for the business users that commission them. IT management would like a platform that enables solutions that are quick to build and deploy, easy to modify and extend, and align with business needs. Business users would like a platform that enables rapid decision making; helps the business sense and responds to changing market conditions; and aligns business needs with IT assets. Solutions would have to provide rapid time-to-benefit, in line with the speed at which market conditions change.
The implications and benefits for both business users and solution providers can be summed up in three words: agility, adaptability, and alignment. These three traits are the central themes of an article on supply-chain management called "The Triple-A Supply Chain," by Prof. Hau L. Lee, in the October 2004 edition of the Harvard Business Review. In this article, Prof. Lee makes the case that "[t]he best supply chains aren't just fast and cost-effective. They are also agile and adaptable, and they ensure that all their companies' interests stay aligned." It is the thesis of this book that the traits of agility, adaptability, and alignment are requirements on all enterprise solutions, and not just in the supply-chain management space. Furthermore, composition is a means to these ends, and this book will show how to architect these kinds of composite applications on the 2007 Microsoft Office System.
Figure 4. Triple-A benefits delivered by self-service architecture
A composite application is a collection of software assets that have been assembled to provide a business capability. These assets are artifacts that can be deployed independently, enable composition, and leverage specific platform capabilities.
Figure 5. High-level representation of a composite application
In the past, an enterprise's software assets were usually a set of independent business applications that were monolithic and poorly integrated with each other. However, to get the business benefits of composition, an enterprise must treat its software assets in a more granular manner, and different tiers of architecture will require different kinds of assets, such as presentation assets, application assets, and data assets. For example, a Web service might be an application asset, an OLAP cube might be a data asset, and a particular data-entry screen might be a presentation asset.
An inventory of software assets by itself does not enable composite applications. This requires a platform with capabilities for composition—that is, a platform that provides the ability to deploy assets separately from each other, and in combination with each other. In other words, these assets must be components, and the platform must provide containers.
Containers provided by the platform must be of different types, which map to the different tiers in the architecture. Enterprise architectures are usually decomposed into three tiers: presentation, application (or business logic), and data. So, the platform must provide containers for these. However the three-tier architecture assumes structured business processes and data, where all requirements are made known during the process of designing and building the system. By their very nature, composite applications presume that composition of solutions can occur after assets have been built and deployed, and so need to account explicitly for people-to-people interactions between information workers that are essential to get any business process complete. Usually, these interactions are not captured by structured processes or traditional business applications, and therefore it is critical to add a fourth tier—the productivity tier—to account for these human interactions. This is shown in Figure 6.
Figure 6. The four tiers of a composite application (Click on the picture for a larger image)
Traditional discussions around the architecture of business applications tend to focus on the application tier as being the connection between people and data. Typically, however, the application tier contains structured business logic, and this holds for discussions around Service-Oriented Architectures (SOAs), Enterprise-Service Buses (ESBs), Service-Component Architectures (SCAs), or most other architectural perspectives in the industry today—including first-generation discussions around composite applications. However, building a composite application requires a mind-set that not only is the productivity tier a critical element of the stack, but also contains the most business value.
To expand on the comparison between composite applications and SOA, both of them target flexibility and modularization. However, SOA provides flexibility at just one tier: the structured business logic in the middle tier. Composite applications target flexibility at all four tiers. That said, a composite application is a great way to surface information out of an SOA, and having line-of-business (LOB) applications exposed as services makes it easier to build support for cross-functional processes into a composite application.
Therefore, to design a composite application, a solutions architect must:
- Choose a composition stack. Pick one or more containers from each tier, and a set of component types that are deployable into those containers.
- Choose components. Define the repository of assets that must be built from this set of component types, based on business needs.
- Specify the composite application. Define the ways in which those assets will be connected, to provide a particular cross-functional process. The platform should enable these connections to be loosely coupled.
Then, after deployment, users will have the opportunity to personalize both assets and connections, as the composition stack should enable this through loose coupling and extensibility mechanisms.
A figurative representation of a composite application is shown in Figure 7, which shows a very abstract representation of an enterprise solution, deconstructed along the lines of Figure 6.
At the top are information workers, who access business information and documents through portals that are role-specific views into the enterprise. They create specific documents during the course of business activities, and these activities are part of larger business processes. These processes coordinate the activities of people and systems. The activities of systems are controlled through process-specific business rules that invoke back-end LOB applications and resources through service interfaces. The activities of people plug into the process through events that are raised when documents that are specific to the process are created or modified. Then business rules are applied to the content of those documents, to extract information, transform it, and transfer it to the next stage of the process.
Figure 7. Deconstructing an enterprise application
Today, most LOB applications are a collection of resources, hard-coded business processes, and inflexible user interfaces. However, based on the previous section, it is clear that enterprise solutions must be broken down into a collection of granular assets that can be assembled into composite applications. A high-level approach for doing this to any business process is listed in Table 1, and subsequent chapters will elaborate upon these steps:
- Decompose the solution for a business process into software assets corresponding to the elements shown in Table 1.
- Package all assets corresponding to a given business process into a "process pack" for redistribution and deployment. This would contain metadata and software components, as well as solution templates that combine them. The process pack would also contain service-interface definitions that would enable connections to other IT systems. These connections would be enabled by implementing the service interfaces, for example, to connect to LOB applications and data. The goal is to be able to layer a standardized business process easily onto any heterogeneous IT landscape.
- Deploy the process pack onto a platform that provides containers for the types of assets into which the solution has been decomposed. The platform should provide capabilities for rapid customization, personalization, reconfiguration, and assembly of assets.
- Connect the assets within the process pack to existing LOB systems and other enterprise resources by implementing the services interfaces. These connections could be made using Web services technologies, other kinds of custom adapters, or potentially even Internet protocols like RSS.
Table 1. List of application assets for composition
Deployment of enterprise applications should be tied to business benefits in the Triple-A sense (agility, adaptability, alignment). These benefits must be demonstrated from two perspectives:
- Solution-provider perspective (or Development perspective)—This is the perspective of the organization that builds an enterprise application. This might be an Independent Software Vendor (ISV), a Systems Integrator (SI), or even an in-house IT department. The solution-provider perspective is concerned primarily with benefits gained in activities relating to designing, implementing, and deploying enterprise applications.
- Solution-consumer perspective (or User perspective)—This is the perspective of the organization that uses an enterprise application. Typically, this is the business unit that commissioned the enterprise application. The solution-consumer perspective is concerned primarily with benefits gained by the business after the solution has gone into production.
The benefits of composition that can be reasonably expected in each of these two perspectives are listed here, along with some high-level best practices to achieve these expected benefits.
Composition must promote alignment among key stakeholders. This could be internal alignment (align different groups within the enterprise) or external alignment (align with suppliers, customers, and partners).
- Solution-provider perspective: Solutions should align existing IT assets with the needs of business owners.
- Solution-consumer perspective: It should be easy to assemble composite applications that align groups within a company, and also across organizational boundaries. Frequently, this will require applications that support cross-functional processes and enable collaboration.
In-process documents should be stored alongside the business processes that they support. These might be explicitly defined processes or ad-hoc collaboration. Policies should be in place to enforce document life-cycle management.
Business processes that are explicitly modeled (as opposed to ad-hoc collaboration) should be modeled using artifacts that are first-class citizens of the development process. This implies that business process definitions are always in sync with the solution, and that these process definitions can become the basis for alignment between business and IT groups.
Business processes within an enterprise must be aligned with business processes implemented by customers, suppliers, and partners. This requires a shared understanding and agreement on public processes among the different organizations. Public processes are those business processes that define the organization's behavior that is visible from the outside world—for example, interactions with trading partners. Private processes are those business processes that are internal to the organization. Figure 8 more clearly illustrates this.
SLA information should be clearly defined for public processes. Integration requirements should be spelled out in the contract.
There should be appropriate visibility across organizational boundaries. This requires exchanging appropriate data—not just transactional data, but also reports, analyses, status, and exception information from business processes. Reports and metrics should be in place to provide incentives that promote better performance—within the organization, as well as partner performance.
As data flows across organizational boundaries, data ownership should be spelled out, and data quality should be required, measured, and enforced.
Integration across organizational boundaries should be done through adoption and commitment to relevant public standards. These standards might be for wire protocols (WS-* standards) or document types (RosettaNet). Standards-based integration can usually lead to more maintainable solutions, as well as to cost savings.
Create deep visibility and alert capabilities to capture events that occur both within the enterprise and outside it (for example, relating to trading partners).
Figure 8. Differentiating between public and private processes (Click on the picture for a larger image)
Composition must reduce time-to-response for the business when markets change.
- Solution-provider perspective: The platform for composition should provide adaptive range. This means that it should be easy to externalize points of variability within the solution, so that those artifacts can be changed easily without ripple effects across the rest of the solution. For example, this might mean changing business rules, user interfaces, and business processes.
- Solution-consumer perspective: It should be easy to reconfigure composite applications when the business must respond to changing market conditions.
- The enterprise architecture should ensure that the implementation of public processes is cleanly separated from the implementation of private processes, as shown in Figure 8. This allows private processes to change without affecting integrations with trading partners. Conversely, it allows changes by trading partners to be isolated from internal systems.
- Avoid hard coding of policies, configuration, rules, or any other kinds of business metadata. Abstract legacy application functions into service interfaces that can be plugged into any kind of business process. Explicit business process models should be separate from service implementations.
- It is not necessary to capture all person-to-person interactions in process models, as the platform should enable ad-hoc document routing. It should be possible for business users to set up these interactions easily without a lot of involvement from IT, but in a way that enterprise document life-cycle policies can be enforced.
- Avoid tight coupling of business processes to trading partners. Integration protocols, interfaces, and message formats should depend only on the public processes of your partners, and not their private processes.
- Avoid tight coupling of systems, such as through point-to-point interfaces and connections. Reducing system dependencies increases the flexibility to make changes.
- Avoid tight coupling of user interfaces to back-end systems. Systems might have to change, but it is hard to retrain users. Similarly, it should be easy to change user interfaces in response to business needs, without large-scale changes to back-end systems.
- Abstract business process workflows from business service implementations. Also abstract business rule definitions from workflows. It should be easy to change a business rule without changing process definitions, and it should be easy to change process definitions without changing the way a business service is implemented.
- Organizations should differentiate between those business processes that are commoditized and those processes in which innovation through new technologies and business models can provide competitive advantage. Organizations should seek packaged solutions and/or leverage industry-standards work for commoditized processes. The organization should be willing to deploy custom or semi-custom solutions for those business processes in which they seek advantages through process innovation.
Composition should reduce the time-to-benefit of enterprise applications by reducing the time-to-deployment of end-to-end solutions, reducing development and deployment costs, and leveraging best practices. Also, when the business must respond to changes quickly or handle external disruptions smoothly, it should be easy to modify composite applications to accomplish this.
- Solution-provider perspective: Assets that make up a composite application should leverage established industry standards that make the solution quick to build and easy to assemble together.
- Solution-consumer perspective: A deployed enterprise solution should enable quick decision making by business decision makers through real-time data flows and business-intelligence tools in the context of the automated process
- Choose a technology platform that reduces the time to build and deploy composite solutions. This should provide a wide range of component types and containers, so that a rich variety of composite applications can be assembled, depending on the needs of the business process. The platform should also provide the tooling for composition.
- Decompose enterprise solutions into more granular software assets, so that solutions can be assembled, instead of developed from scratch.
- Make it possible to plug business services into—and unplug them from—the platform for assembling composite applications. This will require that existing applications be exposed through service interfaces, and that there be a mechanism for governance that enables client applications to register and bind to relevant business services. For those services in which it is not necessary to track clients, publish service interfaces that can be subscribed to by composite applications.
- Enable real-time (or near real-time) information flows between the enterprise, its customers, suppliers, and partners (such as logistics providers). Make use of up-to-date data to automate collaborative business processes with customers, suppliers, and partners to reduce response times when change occurs.
- Avoid tight coupling of business processes to specific applications. Business processes must be fluid, but is hard to reconfigure an application beyond the boundaries set by the application vendor.
- Reuse existing IT assets by connecting into what is already there, extracting more value from what is already there, and finally extending and evolving it.