An Overview of Service-Oriented Architecture in Retail
Service-Oriented Architecture (SOA)
Summary: In this changing business climate, where globalization and demanding customers require agility and flexibility, retailers' IT systems are not helping them in meeting these expectations. Most of the legacy retail systems were built following tightly coupled point-to-point integration principles. Service orientation offers great benefits in moving from these legacy systems to more flexible and agile systems. This article looks at the challenges faced by retailers, what is service orientation, and how it can benefit the retail enterprises. (12 printed pages)
To stay competitive in this globalized world, retailers should be able to launch products and promotions quickly and provide enhanced customer service. All of these factors increase the need for flexibility and agility in retail applications. However, at a time when competition is keenly interested in buzzwords like agility and adaptability, businesses are confronted with systems that are seemingly built of steel and cement, developed as though things would never change. These systems become more expensive and cumbersome each time organizations tweak them to fit new business imperatives. Unfortunately, this leads to less automation, as systems require constant human intervention and IT work. All too often, the alignment of technology with business objectives gets bogged down by integration problems and progresses no further than the white board. One of the common areas where retailers face huge challenges is the availability of the right information, at the right time, and at the right place—for example, making sales information available in near real time to the store managers, regional managers, and merchandising managers in the corporate headquarters. Making information available in near real time requires systems that can generate near real-time data, send it to the right places, and consume it in near real time.
Service orientation addresses these challenges by centering on rapidly evolving XML and Web services standards that are revolutionizing how developers compose systems and integrate them over distributed networks. No longer are developers forced to make do with rigid and proprietary languages and object models that used to be the norm before service orientation came into play. The emergence of this new methodology is helping to develop new approaches specifically for Web-based distributed computing. This revolution is transforming the business by integrating disparate systems to establish a real-time enterprise.
Today's retail business climate is defined by constant change and increasing competitive pressures. As retailers strive to maximize business results through growth and increasing profit margins, they face more demanding customers, new regulations, globalization, and the destabilizing effects of technological advancement. All of these critical factors are changing the retail landscape significantly ands introducing new challenges and generating new requirements for the retailers. Remaining competitive in this environment requires IT investments to increase employee impact and drive business value, while keeping a tight control on costs. Retailers seek to leverage their existing siloed applications through integration. Historically, integration has been done using proprietary protocols requiring cumbersome application connectivity—tight coupling. Such tightly coupled solutions make it difficult for applications to adapt to changing business requirements as each modification to one application may force developers to make changes to other tightly coupled applications.
To achieve full connectivity between disparate applications, retailers must share information such as transaction data, out-of-stock notification, alerts, and many others in real time, instead of simply coordinating data movement. All of these require complex transformations normalizations and movement of data. Service orientation promises to help retailers in integrating existing applications and components and to make data available for reuse. Retailers need service-oriented architectures (SOA) that incorporate enterprise-class data transformation and integration services to address complex data integration requirements such as data synchronization and information hubs. To ensure maximum uptime and avoid potentially costly and disastrous network outages, retailers—from Small and Medium Businesses (SMB) to the largest enterprises retailers—must address these integration issues before it is too late. Only a few top global enterprise retailers have achieved the level of integration that provides them with visibility into the enterprise in real time. However, with the changing landscape, other enterprise retailers must get on board to achieve the same—if not better—integration across their enterprise.
Retailers are under relentless pressure to streamline processes, survive shrinking profits, shepherd inventory, summon up innovative sales and marketing programs and share information in real-time across store, channel and system boundaries. Retailers must:
- Create rich customer experience. Creating engaging customer experience deals with how quickly and smoothly an item can be checked out, availability of information for the customer, right products on the right shelf, and so forth. These issues are dependent on the quality of in-store solutions such as in-store. Also, a real-time inventory tracking system will alert the staff to order and/or restock the shelf based on the sales data.
- Develop a transparent, collaborative, real-time supply chain. This issue is related to the ability of the enterprise retailer to manage inventory and orders in real time to optimize the inventory stock and minimize the out of stock situation, or Just-in-Time (JIT) inventory.
- Offer multichannel data management, especially with regard to their call center. As devices proliferate in the stores and broadband availability grows, the gap between different retailing channels is narrowing. This is related to providing customers the ability to order a product over the phone or Web and pick it up from the store. The experience should be seamless to the end customer.
- Consider price management. This issue is related to the ability of retailers to upload price updates seamlessly to all the stores. This can include promotional price updates, new item introductions, and so forth.
- Consider point-of-sale (POS) transformation. Typically, attitude of the retailers is that the current POS platform works or it is good enough. However, the critical factors ignored by the retailers are the cost and impact this legacy system has on the overall enterprise. Increasing maintenance costs, support costs, and the limitations posed by the legacy systems for marketing and merchandising are completely ignored when thinking about the POS systems. So, retailers are now forced to rethink about the POS systems and must move from a traditional point-of-sale to a point-of-service system, which is more flexible and supports modern store technologies such as customer loyalty, self-service, and advanced payment options.
These critical challenges in the retail industry can be summarized as: boost sales, reduce costs, turn inventory faster, and share knowledge rapidly. To stay competitive, retailers must address these challenges by building a real- time enterprise. In summary, shortcomings of current retail scenarios are:
- Siloed applications both within the store, within the enterprise, and between the store and the enterprise.
- Tightly coupled integration.
- Huge latency in information exchange.
- File-based communication.
- Single-threaded communication.
- Limitation on how quickly retailers can react to customer complaints.
- Limitations on adapting new and emerging technologies.
- Lack of visibility to the store manager on key performance indicators (KPIs).
As the industry evolves, the edge of the retail enterprise will be the focal point of integrated innovation and the redefinition of value offer to customers. Through integrated innovation, technology can enable an enterprise retailer to get closer to its customers in the store and drive value from that interaction. To this end, Microsoft has created the Smarter Retailing Initiative (SRI). A white paper on SRI provides details of this initiative (see References).
Service orientation (SO) is the natural evolution of current development models. The, '80s saw object-oriented models; then came the component-based development model in the '90s; and now we have service orientation (SO). Service orientation retains the benefits of component-based development (self-description, encapsulation, dynamic discovery, and loading), but there is a shift in paradigm from remotely invoking methods on objects, to one of passing messages between services. Schemas describe not only the structure of messages, but also behavioral contracts to define acceptable message exchange patterns and policies to define service semantics. This promotes interoperability, and thus provides adaptability benefits, as messages can be sent from one service to another without consideration of how the service handling those messages has been implemented.
Figure 1. Service interaction
Service orientation provides an evolutionary approach to building distributed software that facilitates loosely coupled integration and resilience to change. With the advent of the WS-* Web Services, architecture has made service-oriented software development feasible by virtue of mainstream development tools support and broad industry interoperability. Although most frequently implemented using industry standard Web services, service orientation is independent of technology and its architectural patterns and can be used to connect with legacy systems, too.
Unfortunately, the benefits offered by service orientation and SOA have been obscured by the hype and confusion that increasingly surround the terms. As awareness and excitement around SOA have swelled, the clear lines that once defined service orientation have been blurred. However, SO does offer some specific benefits when utilized for the right purpose. There are three important observations about SO:
- It's evolutionary. The principles of service-oriented development build on decades of experience in building real world distributed applications. SO incorporates concepts such as self-describing applications, explicit encapsulation, and dynamic loading of functionality at run time—principles first introduced in the 1980s and 1990s through object-oriented and component-based development. What changes with SO is the metaphor with which developers achieve these benefits. Instead of using method invocation on an object reference, service orientation shifts the conversation to that of message passing—a proven metaphor for scalable distributed software integration.
- It's not a product or technology. It is a set of architectural principles expressed independently of any product. Just as development concepts such as polymorphism and encapsulation are independent of technology, so is service orientation. And while Web services have in recent years facilitated the development of service-oriented applications, they are not required to do so.
- Incremental. Finally, service orientation can and should be an incremental process—one that can often be done in-house. Customers should not be required to dramatically re-engineer their businesses to attain the benefits of service orientation. Instead, they should be able to leverage existing IT assets in doing so. Service-oriented development can often be achieved using the skills and technologies customers already have today.
The fundamental building block of service-oriented architecture is a service. A service is a program that can be interacted with through well-defined message exchanges. Services must be designed for both availability and stability. Services are built to last while service configurations and aggregations are built for change. Agility is often promoted as one of the biggest benefits of SOA—an organization with business processes implemented on a loosely coupled infrastructure is much more open to change than an organization constrained by underlying monolithic applications that require weeks to implement the smallest change. Loosely coupled systems result in loosely coupled business processes, since the business processes are no longer constrained by the limitations of the underlying infrastructure. Services and their associated interfaces must remain stable, enabling them to be reconfigured or re-aggregated to meet the ever-changing needs of business. Services remain stable by relying upon standards-based interfaces and well-defined messages—for example, using SOAP and XML schemas for message definition. Services designed to perform simple, granular functions with limited knowledge of how messages are passed to or retrieved from it are much more likely to be reused within a larger SOA infrastructure.
There are "four basic tenets" of service orientation, which must be followed while designing services. The four basic tenets are:
- Boundaries are explicit.
- Services are autonomous.
- Services share schema and contract, not class.
- Service compatibility is determined based on policy.
Service orientation does not necessarily require rewriting functionality from the ground up. Following the four tenets enables reuse of existing IT assets by wrapping them into modular services that can be plugged into any business process that you design. The goals for doing this should be to:
- Connect into what is already there. Layer business process management, collaborative workflows, and reporting on top of existing IT assets.
- Extract more value from what is already there. Enable existing applications to be reused in new ways.
- Extend and evolve what we already have. Create IT support for new cross-functional business processes that extend beyond the boundaries of what the existing applications were designed to do.
One of the key benefits of service orientation is loose coupling. No discussion of Web services seems complete without some reference to the advantages of looser coupling of endpoints (applications) facilitated by the use of Web service protocols. The principle is that of using a resource only through its published service and not by directly addressing the implementation behind it. This way, changes to the implementation by the service provider should not affect the service consumer. By maintaining a consistent interface, the service consumer could choose an alternative instance of the same service type(for example, change service provider) without modifying their requesting application, apart from the address of the new instance. The service consumer and provider do not have to have the same technologies for the implementation, interface, or integration when Web services are used (though both are bound to use the same Web service protocols).
What kinds of services should we create? One way of organizing services is as follows:
- Infrastructure services.
- Data services—Simple atomic operations on an entity.
- Activity services—Coordinate data services for business process execution.
- Process services—Long-running business processes, possibly complex workflow, and human interaction.
- Event services—Notify subscribers of events.
Figure 2. Sample business services
It is not necessary to create all of these types of services. A better way would be to expose services incrementally, in the context of well-defined projects that create business value. Figure 2 shows a set of sample business services that could be deployed in the enterprise.
When designing business software, we should remind ourselves that the objective is delivering agile systems in support of the business; not service orientation (SO). Instead, SO is the approach by which we can enable business and technology agility, and is not an end in itself. This must particularly be borne in mind with references to Web services. Achieving the agility that so often accompanies Web services is not just a consequence of adopting Web service protocols in the deployment of systems, but also of following good design principles. In this article, we consider several principles of good service architecture and design from the perspective of their impact on agility and adaptability.
There are many benefits of service orientation. For example, SO:
- Enables new opportunities in all industries.
- Helps in multichannel integration, and provides seamless customer experience across channels.
- Reduces integration costs.
- Reduces duplication of effort.
- Provides real-time integration with suppliers and partners.
- Provides real-time decision making.
There are certainly some myths associated with service orientation, which are very important to understand before taking a dive into this new methodology. The table below describes some of the top myths surrounding service orientation and the facts behind them.
|SOA is a technology.||SOA is a design philosophy independent of any product, technology, or industry trend.|
|SOAs require Web Services.||SOAs can be realized using Web services, but using Web services will not necessarily result in a SOA.|
|SOA is new and revolutionary.||EDI, CORBA, and DCOM were conceptual examples of SO.|
|SOA ensures the alignment of IT and business.||SOA is not a methodology.|
|A SOA Reference Architecture reduces implementation risk.||SOAs are like snowflakes: No two are the same.|
|SOA requires a complete technology and business-processes overhaul.||SOA should be incremental and built on your current investments.|
|SOA requires an army of consultants.||SOA requires tools, not consultants.|
|We need to build a SOA.||SOA is a means, not an end.|
From the above, it is clear that this new paradigm does carry huge benefits and does help in achieving flexibility and agility in the business processes. This section will focus on how SO helps in the retail business processes and what benefits can be obtained from it. Retail enterprises are facing some of the same challenges that other industries face. Current retail solutions pose many challenges in providing a seamless customer experience across multiple channels and providing information where it is needed:
- Siloed applications with minimal integration between them
- Tightly coupled point-to-point integration
- Lack of real time business intelligence software that provides real-time key performance indicators (KPI) for decision making
- Duplicate customer data—each channel maintaining its own customer data
- Lack of channel integration
- Lack of applications that can pull together the information gleaned from stores and other channels
These top challenges are inhibiting the retailers from providing enhanced customer experience. Traditionally, when it comes to integrating internal applications like order management, inventory management, and customer databases, retailers have lagged other industries in the adoption of standards like Web services, XML, and BPEL, opting instead to build custom links between applications. According to an analyst report, as of 2005, retailers had created fewer Web services than any other industry. Therefore, unlike other industries, retail solutions are predominantly either homegrown or legacy applications, which cannot be easily replaced or require huge investments to replace. Starting by incrementally wrapping the existing IT investments into modular services will help in plugging them into new business processes. Another huge advantage it provides is the ability to incrementally replace these legacy applications with new applications with minimal integration challenges and the least disruption to the business operations when such replacement becomes economically feasible.
Retail enterprises applications such as inventory management, purchasing, merchandising, human resources, customer relationship management, data warehousing, and business analytics typically communicate with one another over a point-to-point tight integration. This takes away the flexibility that retailers need to change business processes to meet reality of the globalization and customer demands. This is especially true considering the migration of manufacturing to the Far East. With this development, supply chain management with suppliers half way around the world requires the ability to quickly change business processes to integrate with new suppliers over new technology. Those retailers that are able to manage and change their internal systems quickly are able to enjoy the fruits of their work in terms of growth. Other retailers that are struggling with these challenges are falling behind in revenues.
As mentioned before, one way to bridge this gap and achieve the flexibility and agility needed in this current environment is to first wrap existing applications into modular services. This can help in plugging them into any new business process that is created. However, before going into those steps, first look at a typical integration between existing applications. Figure 3 shows how different applications in a retail enterprise integrate point-to-point with other applications. This leads to a one-to-many type of integration, which is expensive to maintain. Every time one of the applications changes, all of the integration points must be revisited, recoded, recompiled, and redeployed.
Figure 3. Implications of point-to-point integration (Click on the picture for a larger image)
So, the first step of moving this point-to-point integration to more of a services oriented integration is to wrap the applications intro modular services. Figure 4 shows the same exiting applications being service enabled, so that many disparate applications and any new applications can easily interact with them. There is guidance (see References) on what the granularity of service should be, which functionality should be first targeted, and so on. One of the design criteria is to identity a finite set of functionality that is shared by many applications, and wrap it with modular services to achieve maximum benefit.
Figure 4. Wrapping legacy applications as services (Click on the picture for a larger image)
This critical first step of service enablement helps in reducing integration costs and also opens up many new opportunities that were not possible before. For example, an external service which is not meeting the service quality agreements can be internalized quickly and easily. The reverse can be done as well, an internal service that is not meeting the service quality metrics can be outsourced. New products and services which were next to impossible before can now be offered to the customers. This service enablement abates the step of creating new dynamic business processes. Traditional point-to-point, tightly coupled software integration forced business processes to be inflexible and brittle. In other words, IT systems dictated the business process as opposed to the other way around. IT systems should empower business processes, enable creation of new business processes and abate overall growth of the business. This is exactly what service orientation intends to do.
Once the technology providers are service-enabled (which is called the business-services layer), retailers can create dynamic business processes using these services. These flexible and agile business processes can include both internal and external services. The steps of services enablement and business process creation lead to what is known as the four-tier architecture with the last tier being the presentation tier. Figure 5 shows an example of a four-tiered architecture for a retail enterprise.
Figure 5. Example of four-tiered retail architecture (Click on the picture for a larger image)
The presentation tier is critical as that is where information worker interaction happens with the systems, and that is where decisions are made. The business-services layer and the business process layers help create a seamless rich user experience across many channels, ranging from Web-based portals to smart clients. Service wrappers in the technology enabler layer provide coarse-grained interfaces for user application consumption, creation of mash-ups, human workflow initiation, creation of composite applications, personalized interfaces, portals, business intelligence dashboards, reporting and other forms of content aggregation. All of these benefits—in fact, the creation of the entire stack—can be achieved by the service enabling of the basic systems and applications.
In retail, there are many opportunities to create services that can be used through multiple channels. Also, they can be offered as independent services to other businesses. Some examples that can reduce duplication, improve business processes, and bring in new revenues are:
- Payment-processing service.
- Lottery service.
- Inventory service.
- Ordering service.
- Returns-processing service.
- Create-invoice service.
- Alerting service.
- Supplier-collaboration service.
- Customer-loyalty service.
All these example services when created correctly can help with new opportunities, better integration of different channels, and ultimately in providing better service. Especially as the customers become more connected and mobile, they prefer to shop online by checking product availability using mobile devices, but picking it up from the store. Meeting this type of customer demand requires real-time integration between different channels and real-time service integration and information delivery. Any retailer that is able to provide this kind of functionality will continue to gain customer loyalty and enjoy increased revenue. Figure 6 shows an example of potential new services that can be offered to customers as part of an integrated seamless experience to customers.
Figure 6. Example of potential new services (Click on the picture for a larger image)
For example, retailers can offer customer loyalty across multiple channels through the loyalty service which can either integrate with an internal loyalty systems or external loyalty providers. Also, retailers can offer banking services at the point-of-service (POS) terminals or provide ability for customers to return items at any store irrespective of where the item was bought. This type of enhanced customer experience will build customer loyalty and increase revenues. Similarly, a price check service can help customers check prices of items without seeking a sales agent directly using their cell phone. A camera on the phone can take a picture of the bar-code and SMS it to a centralized price-check service which instantly returns the price and also availability of items in stock at nearby stores. Possibilities are huge once the services layer is implemented correctly.
As retailing expands beyond the single channel of brick and mortar, cross-channel integration becomes key to providing seamless experiences to customers. To get better product visibility across channels and to identify customers across channels, retailers need the ability to integrate the siloed channels. Service orientation makes it easy to achieve this. Figure 7 shows integration of legacy applications with different channels, irrespective of whether they are the existing channels or new channels. The goal should not just be integration but achieving real-time integration as it is the best way to provide seamless experience to the customers.
Figure 7. Integrating legacy applications with various channels (Click on the picture for a larger image)
In summary, service orientation offers many benefits to retailers, some of which are that SO:
- Eases integration of disparate systems.
- Opens up new opportunities.
- Enables retailers to offer enhanced customer service.
- Helps build loyalty.
- Reduces the overall cost of integration and maintenance.
While service orientation provides a means to building integrated retail systems, it is insufficient by itself. Success depends not only on service delivery, but also on the ability to consume services in a rich, meaningful way. Service consumption needs to be contextual, mapping to the natural workflow of employees, customers, and partners. To that end, composite applications deliver an integrated user experience that spans smart clients, Web applications, and mobile devices.
Figure 8. Composite applications combine context with service orientation.
So, what are composite applications? As figure 8 shows, composite applications are a collection of software assets that have been assembled to provide a business capability. These assets are artifacts that can be deployed independently, enable composition, and leverage specific platform capabilities. And what are these software assets that enable composition? In the past, a retailer's software assets were usually a set of independent business applications (such as merchandising application, inventory application, and so forth) that were monolithic and poorly integrated with each other. However, to get the business benefits of composition, an enterprise must treat its software assets in a more granular fashion, and different tiers of architecture will require different kinds of assets. This includes things like presentation assets, application assets, and data assets. For example, a Web service might be an application asset, an OLAP cube might be a data asset, and a particular data-entry screen might be a presentation asset.
Having an inventory of software assets by itself does not enable the ability to assemble composition applications. This requires a platform with capabilities for composition—that is, a platform that provides the ability to deploy assets separately from each other, and in combination with each other. In other words, these assets must be components, and the platform must provide containers.
Containers provided by the platform must be of different types, which map to the different tiers in the architecture. Enterprise architectures are usually decomposed into three tiers: presentation, application (or business logic), and data. So, the platform needs to provide containers for presentation assets, application assets, and data assets. However, the three-tier architecture assumes structured business processes and data, where all requirements are made known during the process of designing and building the system. By their very nature, composite applications presume that composition of solutions can occur after assets have been built and deployed—and so must account explicitly for people-to-people interactions between information workers that are essential to get any business process complete. Usually, these interactions are not captured by structured processes or traditional business applications; therefore, it is critical to add a fourth tier—the productivity tier—to account for these human interactions.
Figure 9. Composite application tiers (Click on the picture for a larger image)
Traditional discussions around the architecture of business applications tend to focus on the application tier as being the connection between people and data and this holds for discussions around SOAs, Service Component Architectures (SCAs), or most other architectural perspectives in the industry today, including first-generation discussions around composite applications. Typically, however, the application tier contains structured business logic. However, building a composite application requires a mindset that not only is the productivity tier a critical element of the stack, but also it is where the most business value to enterprises can be found.
To expand on the comparison between composite applications and SOA, both of them target flexibility and modularization. However, SOA provides flexibility at just one tier: that of the structured business logic in the middle tier. Composite applications target flexibility at all four tiers: presentation, productivity, application, and data. So, composite applications provide business value to the end users, in a way that goes far beyond what can be achieved with just service orientation. That said, a composite application is a great way to surface information out of a SOA, and having line-of-business (LOB) applications exposed as services makes it easier to build support for cross-functional processes into a composite application.
Therefore to summarize, the ability to build and deploy composite applications requires a platform that provides containers and component types for each of the four tiers listed in Figure 9. The solutions architect must then:
- Choose a composition stack. Pick one or more containers from each tier, and a set of components types that are deployable into those containers.
- Choose components. Define the repository of assets that must be built from this set of component types, based on business needs.
- Specify the composite application. Define the ways in which those assets will be wired together, to provide a particular cross-functional process.
The 2007 Microsoft Office system is a platform for building such composite applications, which are called Office Business Applications (OBAs). At a very high level, some of the technical capabilities of the 2007 Microsoft Office System are summarized in table below. Each of these capabilities is a powerful feature when looked at individually; however, it is the combination of these different technologies into a single integrated platform that makes composition practical. It is the integration of these technologies that enables delivery and deployment of composite applications without an overwhelming increase in complexity in the overall platform, tooling, and architecture.
|Web site and security framework||A common framework for creating different kinds of sites such as team collaboration sites, intranet portals, Internet Web sites.|
|Open XML file formats||Open formats to represent business documents that can easily be read, transformed and visualized. This enables rich server-side processing of documents in ways that was not possible before. With prior versions of Microsoft Office, parsing the document using the object model required an instance of the client application.|
|Extensible UI||Server-side portal that can be extended by users from a catalog of Web Parts, and the catalog itself can be extended by solutions providers. Client applications with rich capabilities for extensibility through Visual Studio Tools for Office.|
|Business Data Catalog||A metadata repository to define business entities stored in back-end data stores, to model relationships between entities, and to define actions permissible on entities.|
|Enterprise Search||Surface data from various enterprise sources through search.|
|Workflow||Integration with Workflow Foundation to host workflows that represent people-to-people interactions, and that link user interface elements.|
|Enterprise Content Management||Manage diverse content, with one topology for Web, document and records management. Support for document life-cycle management.|
|Business Intelligence||Server-based Excel spreadsheets, plus business information (BI) components (dashboards, reports, Web Parts) built into the portal and connected to SQL Server Analysis Services.|
|Communication and Collaboration||Support for unified communications integrated into the platform.|
In summary, service orientation offers great benefits to the retail enterprise in incrementally changing systems to meet the business demands. Also, service orientation is not a goal, but is a means to achieve the agility in the retail enterprise. The goal should be to build applications that are flexible in not only meeting business demands, but also in creating new business processes based on the changes in the business climate. Once applications are service enabled, consider assembling composite applications to enjoy the full benefits of service orientation.