Lightweight SOAs: Exploring Patterns and Principles of a New Generationof SOA SolutionsJesus Rodriguez Don Demsak December 2009
Summary: This article explores some of the most common challenges of traditional service-oriented architectures (SOAs) and discusses how those challenges can be addressed by using more scalable, interoperable, and agile alternatives that we like to call “lightweight SOAs.”
IntroductionDuring the last few years, we have witnessed how the traditional approach to service orientation (SOA) has constantly failed to deliver the business value and agility that were a key element of its value proposition. Arguably, we can find the causes to this situation in the unnecessary complexity intrinsic of traditional SOA techniques such as SOAP, WS-* protocols, or enterprise service buses (ESBs). As a consequence, alternate lightweight SOA implementations that are powered by architecture styles such as Representational State Transfer (REST) and Web-Oriented Architectures (WOA) are slowly proving to be more agile than the traditional SOA approach.
SOA: Architecting Without ConstraintsSOA has been the cornerstone of distributed systems in the last few years. Essentially, the fundamental promise of SOA was to facilitate IT agility by implementing business capabilities by using interoperable interfaces that can be composed to enable new business capabilities. From an architectural style standpoint, traditional SOA systems share a set of characteristics, such as the following:
Simple enough, right? An ideal SOA infrastructure should resemble Figure 1.
Figure 1. Ideal SOA
We can all agree that Figure 1, at least in theory, represents an ideal architecture for enterprise applications. Unfortunately, large SOA implementations have taught us that the previous architecture is just that: an ideal that is permeated by enormous challenges in areas such as versioning, interoperability, performance, scalability, and governance. These challenges are a direct consequence of the lack of constraints in SOA systems. Architecture styles that do not impose constraints in the underlying domain quite often produce complex and unmanageable systems. Instead of simplifying the capabilities of SOA and focusing on the important aspects—such as interoperability, performance, and scalability—we decided to abstract complexity with more standards and tools. As a result, we have been building systems that present similar limitations to the ones that originated the SOA movement in the first place. One thing that we have learned from Ruby on Rails is the “convention over configuration” or “essence vs. ceremony” mantra. By removing options and sticking with conventions (that is, standard ways of doing something), you can remove extra levels of abstraction and, in doing so, remove unneeded complexity of systems. Embrace the options, when needed, but do not provide them just for the sake of giving options that will be underutilized. Although this article does not attempt to present a forensic analysis of failed SOA initiatives, we think that it is worth highlighting some of the factors that developers should consider when they implement large SOA solutions. Given the extent of this article, we decided to concentrate on the following topics:
The remainder of this article will dive deep into each of these topics and will explore alternative architecture styles that can help developers implement more robust SOA solutions. SOAP and the Illusion of Transport AbstractionThe current generation of SOA solutions evolved around the concepts of the Simple Object Access Protocol (SOAP). This protocol was originally designed to abstract services from the underlying transport protocol. Conceptually, SOAP services can be hosted by using completely different transports, such as HTTP and MSMQ. Although in theory it is a lovely idea, in practice we have learned that SOAP reliance on transport neutrality comes at a significant cost—a cost that can be reduced when you do not need that neutrality. One of the best examples of the limitations of transport neutrality is the use of HTTP with SOAP-based service endpoints. The vast majority of SOAP-based services rely on HTTP as the transport protocol. However, the SOA HTTP binding uses only a very small subset of the HTTP specification, reduced to the POST method and a couple of headers. As a result, SOAP HTTP–based services do not take advantage of many of the features that have made HTTP one of the most popular and scalable transport protocol in the history of computer systems. If we have inherited something good from SOAP, it has been the use of XML, which has drastically increased the interoperability of distributed applications. However we can all agree that SOAP has failed on its original expectations. This about it: SOAP was originally coined as the Simple Object Access Protocol; but, as we all learned, it is not simple, is not about object access, and, arguably, is not a protocol. WSDL AbuseThe Web Service Description Language (WSDL) is one of the fundamental specifications in the SOA portfolio. The purpose of WSDL is to describe the capabilities of a service, such as the messages that it can send and receive or how those messages are encoded by using the SOAP protocol. Conceptually, WSDL represents an evolution of previous description languages, such as the Interface Description Language (IDL), which was the core of distributed programming technologies such as COM and CORBA. Following a similar pattern, WSDL quickly became the fundamental artifact that client applications used to generate “proxy” representations that abstract the communication with the service. Undoubtedly, the idea of generating proxy artifacts that are based on a service Web Services Description Language (WSDL) can facilitate client-service interactions in very simple environments. Like its predecessors, the fundamental challenge of this approach is that it introduces a level of coupling between the client and the service. In large SOA scenarios that involve hundreds of services and clients, that level of service-client dependency is typically the cause of serious service-management and versioning challenges, as Figure 2 illustrates.
Figure 2. WSDL dependency—a big challenge in large SOA implementations
To ESB or Not to ESBThe seamless integration of heterogeneous line-of-business (LOB) systems as part of business processes has been one of the promises of enterprise SOA systems. These integration capabilities are normally powered by a set of common patterns that constitute the backbone of what the industry has considered one of the backbones of SOA: the enterprise service bus (ESB). Although there is no formal industry standard that defines what an ESB is, at least we can find a set of commonalities across the different ESB products. Typically, an ESB abstracts a set of fundamental capabilities such as protocol mapping, service choreographies, line- of-business (LOB) adapters, message distribution, transformations, and durability. Theoretically, we can use this sophisticated feature to abstract the communication between services and system—making the ESB the central backbone of the enterprise, as Figure 3 illustrates.
Figure 3. Central ESB
As architects, we have to love absolutely the diagram in Figure 3. Undoubtedly, it represents an ideal model on which messages are sent to a central service broker and from there distributed to the final services. Unfortunately, if we are working in a large SOA, we are very likely to find that a central bus architecture introduces serious limitations in aspects such as management, performance, and scalability, as our entire integration infrastructure now lives within a proprietary framework. Instead of being a facilitator, an ESB can become a bottleneck for the agility, governance, and scalability of our SOA. Consequently, we are forced to start building applications that do not fully leverage the ESB, and our architecture quickly starts to resemble the diagram in Figure 4.
Figure 4. ESB reality in a large enterprise
WS-* MadnessAfter the first wave of SOA specifications was created, several technology vendors began a collaborative effort to incorporate some key enterprise capabilities such as security, reliable messaging, and transactions into the SOAP/WSDL model. The result of this effort materialized in a series of specifications such as WS-Security, WS-Trust, and WS-ReliableMessaging, which the industry baptized as WS-* protocols. Beyond the undeniable academic value of the WS-* specifications, they have not received a wide adoption in heterogeneous environments. The limited WS-* adoption in the enterprise can ultimately be seen as a consequence of the incredibly large number of WS-* specifications that have been produced during the last few years. Currently, there are more than a hundred different versions of WS-* protocols, from which just a handful have seen real adoption on SOA solutions. Interoperability is by far the most challenging aspect of WS-*–based solutions, as different Web-service toolkits sometimes implement different WS-* protocols, different versions of the same protocols, or even different aspects of the same specification. Additionally, the adoption of WS-* has fundamentally been reduced to the .NET and Java ecosystems, which makes it completely impractical to leverage emerging programming paradigms such as dynamic or functional languages into SOA solutions (see Figure 5).
Figure 5. WS-* interoperability challenges
SOA Governance or SOA DictatorshipManagement and governance must be at the center of every medium to large SOA. While we keep enabling business capabilities via services, it is important to consider how to version, deploy, monitor, and manage them. This set of functionalities has been the bread and butter of SOA-governance platforms that have traditionally evolved around the concepts of the Universal Description, Discovery, and Integration (UDDI) specification. Depending on our implementation, we might find that SOA- governance platforms are sometimes too limited for truly managing complex Web services. These types of challenges are very common in SOA-governance solutions and are a consequence of the fact that Web-service technologies have evolved relatively faster than the corresponding SOA-governance platforms. SOA-governance technologies have traditionally addressed those limitations by relying on a centralized model in which services are virtualized in the governance-hosting environments and policies are enforced at a central location. Although this model can certainly be applicable on small environments, it presents serious limitations in terms of interoperability, performance, and scalability (see Figure 6).
Figure 6. Centralized SOA-governance models—impractical in large SOA implementations
Introducing Lightweight SOAsConceptually, SOAs can be a very powerful vehicle for delivering true business value to enterprise applications. However, some of the challenges that are explained in the previous sections have caused SOA initiatives to become multiyear, multimillion-dollar projects that failed to enable over the promised agility. Despite these challenges, the benefits of correctly enabling SOAs can result in a great differentiator to deliver true business value on an enterprise. However, we believe in a lighter, more agile approach that leverages the correct implementation techniques and emerging architecture styles is mandatory to implement SOA solutions correctly. The remaining sections of this article will introduce some of the patterns and architecture techniques that we believe can help address some of the challenges that were presented in the previous section. Fundamentally, we will focus on the following aspects:
Embracing the Web: RESTful ServicesIn previous sections, the authors have explored various limitations of the fundamental building blocks of traditional SOA systems such as XML, SOAP, WSDL, and WS-* protocols. Although Web Services are transport-agnostic, the vast majority of implementations in the real world leverage HTTP as the underlying protocol. Those HTTP-hosted services should (in theory, at least) work similarly to Web-based systems. However, the different layers of abstractions that we have built over the HTTP protocol limit those services from fully using the capabilities of the Web. To address some of these challenges, SOA technologies have started to embrace more Web-friendly architecture styles, such as the REST. REST has its origins in Roy Thomas Fielding’s PH.D dissertation that states the principles that make the Web the most scalable and interoperable system in the history of computer software. Essentially, REST-enabled systems are modeled in terms of URI-addressable resources that can be accessed through HTTP stateless interactions. Following the principles of REST, we can architect highly scalable services that truly leverage the principles of the Web. Undoubtedly, REST has become a very appealing alternative to SOAP/WS-* Web Services. The use of REST addresses some of the limitations of traditional Web Services such as interoperability and scalability. The following are capabilities of RESTful services:
The simplicity from the consumer perspective and high levels of interoperability of RESTful services are some of the factors that can drastically improve the agility of the next generation of SOA solutions. WS-* InteroperabilityDespite the remarkable academic work that supports the WS-* family of protocols, we can all agree that its adoption and benefits did not reach initial expectations. Interoperability and complexity remain as important challenges that we face in the adoption of WS-* protocols in the enterprise. When it comes to WS-* interoperability, the absolute best practice is to identify the capabilities of the potential consumers of our services. Based on that, we can determine which WS-* protocols are best suited for our particular scenarios. In highly heterogeneous environments, we should considering enabling different service endpoints that support various WS-* capabilities. This approach can drastically improve the interoperability of our services, given that the different clients can interact with the service endpoint that interoperates best with them. For example, let us consider a scenario in which we must secure a Web Service that is going to be consumed by .NET, Sun Metro, Oracle WebLogic, and Ruby clients. In this scenario, we could enable three service endpoints with different security configurations, based on the consumer capabilities, as Figure 7 illustrates.
Figure 7. Pattern of multiple service endpoints
Even in scenarios in which we decide to use WS-* protocols, the technique that Figure 7 illustrates helps us improve interoperability by enabling various endpoints that capture the interoperability requirements of the different service consumers. Lightweight Federated ESBsAs explored in previous sections, a centralized ESB very often is one of the fundamental causes of failed SOA initiatives. The ability to centralize very smart functionalities such as message routing, transformation, and workflows is as appealing as it is unrealistic in medium-to large-enterprise environments. Essentially, by relying on the clean concept of the central service bus, we drastically constrain the options for scalability, specialization, and management of the enterprise solutions that leverage our SOA infrastructure. After several failed attempts to implement centralized ESBs in large organizations, the industry has moved to a more agile pattern in which the functionality is partitioned across multiple lightweight physical ESBs that are grouped as a federated entity. This pattern is commonly known as federated ESB and represents one of the emerging architecture styles for building highly scalable ESB solutions (see Figure 8).
Figure 8. Pattern of federated ESB (Click on the picture for a larger image)
The federated-ESB pattern addresses the fundamental limitations of the centralized-ESB model by partitioning the infrastructure into separate ESB that can be scaled and configured separately. For instance, in this model, we can have a specific ESB infrastructure to host the B2B interfaces, while another ESB is in charge of the financial-transaction processing. This approach also centralizes certain capabilities—such as security or endpoint configuration—that do not impose any scalability limitation on the SOA infrastructure. Lightweight GovernanceThe limited adoption of UDDI in large-scale SOA environments has been a catalyst for the emergence of lighter and more interoperable SOA-governance models that leverage new architecture styles, such as the REST and Web 2.0. Essentially, these new models look to remove some of the complexities that are bound to the centralized, UDDI- based architectures—replacing them with widely adopted standards such as HTTP, Atom, and JSON. One of the most popular new SOA-governance models is the idea of a RESTful Service Repository. In this model, traditional SOA constructs such as service, endpoints, operations, and messages are represented as resources that can be accessed through a set of RESTful interfaces. Specifically, the use of resource-oriented standards such as Atom and the Atom Publishing Protocol (APP) is very appealing to represent and interact with SOA artifacts (see Figure 9).
Figure 9. RESTful registry
This model represents a lighter, more flexible approach to both design and runtime governance. For example, runtime-governance aspects such as endpoint resolution are reduced to a simple HTTP GET request against the RESTful interfaces. The main advantage of this type of governance probably is the interoperability that is gained by the use of RESTful services, which will allow us to extend our SOA-governance practices beyond .NET and J2EE to heterogeneous technologies such as dynamic or functional languages. Welcoming the CloudThe emergence of cloud-computing models is steadily changing the way in which we build distributed systems. Specifically, we believe that the next generations of SOA solutions are going to be a hybrid of cloud and on-premises services and infrastructure components. The influence of cloud computing is by no means reduced to the ability of hosting enterprise Web services in private or public clouds. Additionally, there are specific SOA-infrastructure components that can be directly enabled from cloud environments as a complement to the traditional on-premises SOA technologies (see Figure 10).
Figure 10. Enhancing on-premises SOAs with cloud infrastructures (Click on the picture for a larger image)
Let us look at some examples:
ConclusionThe traditional approach to SOA introduces serious challenges that make it impractical for large implementations. This article suggests a series of patterns that can help developers enable lighter, interoperable, and scalable SOAs that can enable true business agility in large enterprise scenarios. Transport Abstraction
SOAP & WSDLIf you have already decided to standardize on HTTP, there is little need for SOAP and WSDL. Learn to embrace technologies such as REST, JSON, and Atom Pub, and use the Web to the fullest extent.
Governance & DiscoverabilityIf you do not have WSDL, how can you govern your corporate services? By using a service registry, of course! UDDI failed, but that does not mean that a service repository was not needed—just that UDDI was too complex and looking to solve the wrong issues. By using a lightweight service registry that is built upon RESTful services, you can still supply governance and discoverability, without the complexity of UDDI.
Enterprise Service BusTo ESB or not to ESB: That is the question. Point-to-point communications are strongly coupled and easy to implement. But point-to-point communications, by their very nature, are brittle, tend to stagnate, and limit the business-intelligence opportunities that are embedded in the messages.
Cloud-Based ServicesEverything in the cloud: That seems to be where we are headed. Microsoft was a bit early with its My Services concept, but more and more services are headed towards the cloud.
The most important thing to keep in mind when you are building your enterprise services is the mantra “convention over configuration.” By keeping the number of options to a minimum and building only what is required by the business, you can create lighter-weight services that are easier to maintain and enhance.
About the AuthorsJesus Rodriguez ( Jesus.Rodriguez@tellago.com) is the Chief Architect at Tellago, Inc. He is also a Microsoft BizTalk Server MVP, an Oracle ACE, and one of a few architects worldwide who is a member of the Microsoft Connected Systems Advisor team. Don Demsak is a Senior Solution Architect at Tellago, based out of New Jersey, who specializes in building enterprise applications by using .NET. He has a popular blog at www.donxml.com and is a Microsoft Data Platform MVP, and a member of the INETA Speakers Bureau. Follow up on this topic
|
Issue 22 Index
|
.jpg)
.png "Ideal SOA")
.png "WSDL dependency—a big challenge in large SOA implementations")
.png "Central ESB")
.png "ESB reality in a large enterprise")
.png "WS-* interoperability challenges")
.png "Centralized SOA-governance models—impractical in large SOA implementations")
.png "Pattern of multiple service endpoints")
.png "Pattern of federated ESB")
.png "RESTful registry")
.png "Enhancing on-premises SOAs with cloud infrastructures")
