Frederick Chong
with Jim Clark, Max Morris, and Dave Welsh

September 2005

Applies to:
   Enterprise Architecture
   Solution Architecture
   Service Oriented Architecture (SOA)
   Service Oriented Management (SOM)
   Application Integration
   Business Process
   Business Operations Modeling

Summary: The Architecture Chronicles on Dynamic Modeling: Aligning Business and IT seek to present to business, solution, and infrastructure architects a holistic and integrated approach to aligning business and IT through dynamic modeling to achieve better performance, accountability, and business results. This document focuses on the high-level issues of the Web service design for a product shipping solution in the Northern Electronics scenario. (14 printed pages)

Contents

This Document
Abstract
Acknowledgements
Introduction
Developing a Web Services Solution Design
Product Shipping Web Services Solution Design
Designing the Web Services Message Contracts
Handling Business Exceptions
Conclusion
Additional Resources

This Document

This document is part of the Architecture Chronicles on Dynamic Modeling: Aligning Business and IT. This volume seeks to present to business, solution, and infrastructure architects a holistic and integrated approach to aligning business and IT through dynamic modeling to achieve better performance, accountability, and business results. The information map to the series provides an up-to-date description and cross-index of the information available and can be found at the Microsoft Architecture Resource Center.

Abstract

This document describes a high-level solution design for the Web services that enable the automation of certain aspects of the product shipping process at Northern Electronics. It describes how to use different software components in a solution to realize the normal operation and exception flows in a business process. It illustrates how a solution architect can use model-driven design and service orientation approaches to develop a design for a flexible, interoperable solution. The document also illustrates how a solution can provide real-time notification of business operations conditions that need attention without attempting to automate rectification actions that can be unduly complicated and costly to implement.

Acknowledgements

Many thanks to Nelly Delgado for her help with technical writing, Claudette Siroky for her graphics skills, and Tina Burden McGrayne for her copy editing.

The authors would also like to thank Gajanan Phadke, Vishal Kapoor, Amol Wankhede, Aziz Matheranwala, Amit Kumar Srivastav, Bhushan Pawade, Bhasha Johari, Avadh Jain, Mughda Gadre, Maneesha Nalawade,Sonika Arora, Anil Sharma, and Anurag Katre (all of Tata Consulting Services) for their contributions to the implementation of the Northern Electronics solution.

Introduction

This document focuses on the high-level issues of the Web services solution design for Northern Electronics's product shipping solution. Background information about the Northern Electronics scenario can be found in the Architecture Chronicles on Dynamic Modeling: Aligning Business and IT at the Microsoft Architecture Resource Center.

Northern Electronics's products are manufactured in China, shipped to a port in Seattle, Washington, and then transported and stored in a warehouse in Everett, Washington. The products are then picked up and transported to customers across the United States. The company is working to improve the product shipping process, which is part of the broader product pickup and delivery process. Product shipping involves a great deal of coordination. Product shipping includes ordering the transport, moving the right product in the right amount from the right warehouse to the right loading dock, with the right accompanying paperwork, loaded onto the right truck at the right time. Getting everything to work out in product shipping so that the right products are shipped to the customer in a timely and cost-efficient manner is a challenge.

The hand-off of goods from the warehouse involves the coordination and cooperation of partner companies who are transport consolidators. Northern Electronics needs a way to effectively and efficiently communicate with the transport consolidators to coordinate the business activities and to handle problems as soon as they occur. There are many problems that arise quite normally in product shipping. Businesses expect these problems to arise and handle them as they occur. From the perspective of the warehouse, the truck might not arrive on time or the truck might arrive but be the wrong truck. From the perspective of the customer, failure ranges from non-delivery of product, incorrect delivery of the product (the wrong product and/or the wrong amount), delayed delivery of the product—or some combination of these failures. Done poorly, the activities are sequenced wrong and significant delays (and costs) are involved.

Understanding the Product Shipping Process: An Example

Part of Northern Electronics's product line is the manufacturing of electronic parts for remote-controlled airplanes. One of Northern Electronics's customers, Wingtip Toys, assembles remote-controlled airplanes in Dallas, Texas, and then sells the airplanes to retail companies. Northern Electronics hires Acme Consolidation Company, a freight consolidator based in Portland, Oregon, to arrange the transporting of the goods from the warehouse in Everett to the wholesaler's warehouse in Dallas.

In this case, Acme Consolidation Company hires Blue Yonder Truckers, a local trucking company to pick up and transport the goods. Figure 1 describes the high-level business entities involved in the product pickup and delivery scenario.

Figure 1. Entities involved in the product pickup and delivery example

Some of the business activities are internal activities that are relevant only to Northern Electronics. However, other business activities rely on a successful collaboration between Northern Electronics and Acme Consolidation Company.

Who's Who in Northern Electronics

The following individuals from Northern Electronics are involved with designing and implementing the product shipping solution:

• Pam: She is the business program manager whose responsibility in this context is to understand and model the product shipping process.
• Zack: He is the head of IT architecture whose responsibility in this context is to work with Pam to help her understand the IT organization and its requirements.
• Jackie: She is the program manager whose responsibility in this context is to work with Pam, the business program manager, to identify the functional needs of the business users and ensures that the solution meets those requirements.
• Tom: He is the solution architect whose responsibility in this context is to work with Jackie, Pam, and Zack to understand the business and technical requirements. Tom also provides the technical expertise to design the solution.
• Sam: He is the lead developer whose responsibility in this context is to collaborate with Tom to design and implement the solution.

Developing a Web Services Solution Design

Tom's primary responsibility as the architect is to design the solution. To do that, he considers:

• The business operations requirements as provided in specifications by Pam and Zack.
• The non-functional attributes that enable services to be operated, maintained, and upgraded over the course of time.

Tom follows a model-driven development approach that uses highly structured information models to capture the intentions and desired results of stakeholders such as Pam and Zack. In terms of the business operations requirements in particular, this approach ensures that Tom proactively thinks about the instrumentation and tasks required to achieve the desired consequences; he also thinks about actions for the mitigation strategies in the event of business exceptions.

In addition, an overarching mandate that Tom received from the company's executive team is to improve Northern Electronics's ability to remain competitive and grow through the use of information technology. In the near future, the business will need to coordinate with more companies, and specifically, more transport consolidators. Each additional transport consolidator represents another organizational boundary and potentially another platform that Northern Electronics will need to send and receive information in a secure way in real time.

Before designing the solution, Tom reads and thinks a lot about the benefits of a service-oriented architecture. In particular, he decided that it is especially crucial for the solution to observe the four tenets of service-orientation as described in Service Orientation and Its Role in Your Connected System Strategy. Tom also surveys the technology options for implementing the solution. He performs a feasibility study and concludes that Web services are the right building blocks for implementing a service-oriented solution.

Web Service Solution Design: Business Services to Web Services

Tom works with Pam and Zack to identify candidate Web services for the product shipping business collaboration. They review the way shipping is currently done while thinking about what can be improved through automation. Pam and Zack already considered this question from the business perspective when they looked at the business services involved in product shipping. While the business operations specifications capture the detail of that analysis in a formal way, Tom needs the consultation with Pam and Zack to help him get a thorough understanding of what is happening and why the specifications have the requirements they do.

One example of something they review together has to do with when the product shipping business collaboration begins. The negotiation between customers and Northern Electronics is done verbally, usually over the phone. This is outside the scope of the product shipping business collaboration. Moreover, the negotiation is not a good candidate for something that can be automated because the human interaction is probably a necessary aspect of negotiating the purchase. The starting point of the product shipping business collaboration is when the purchase order (PO) is created. The PO is an artifact that can be stored electronically instead of on paper.

Another example of something they review together is the ordering of transport. After the PO is created, the supplier shipping clerk checks to make sure the items are in stock. If the items are in stock, the supplier shipping clerk works with the transport consolidation clerk to coordinate the transport that will pick up the shipment and deliver it to the customer.

Yet another example of something they review together is the security requirements. Identities of the participants in the business collaboration must be represented as digital identities in the Web services domain. The processes of authentication and authorization are necessary to control access, usage, and administration of the services, especially across administrative boundaries—in this case, between the supplier and the transport consolidator. Tom has to make sure he understands issues such as who has the authority to create or change transport order information and who can confirm acceptance of transport. These issues are especially important to meet the business operations requirements, which outline strict legal requirements. For example, they specify the role of who in Northern Electronics is actually allowed to order transport from a transport consolidator (and therefore commit the company to payment for the ordered transport). (Other requirements related to security and identity, such as logging specific activities as they are performed by certain roles, are expressed in the business operations specifications.)

Tom agrees with the conclusion from Pam and Zack that the interactions between the supplier and transport consolidator are good candidates for being automated and implemented using Web services. He reasons that:

• If the transport consolidator has a Web service the supplier shipping clerk can use to send his request to, instead of relying on a phone call, it is unlikely the request will get lost. The supplier shipping clerk can also get proof that the request was received. The transport consolidation clerk can then process the request and generate an acceptance.
• If the supplier has a Web service the transport consolidator can use to send his acceptance of the transport order to, instead of relying on a fax machine, it is similarly unlikely the request will get lost. The acceptance can contain the details and terms of the product pickup. The transport consolidation clerk can get proof that the acceptance was received and thereby that the terms were agreed to.

After Tom examines the business collaboration further, he reasons that the supplier shipping clerk and loading dock clerk can work more efficiently by using an internal Web service to let them know about the shipments that need to be loaded onto the docks in the correct order based on the trucker's expected time of arrival. They can use this internal Web service to record the trucker's credentials and can also use it to confirm that the shipment was picked up. In turn, the internal Web service can then automatically send a pickup confirmation to Acme Consolidation Company's Web service.

Based on this analysis, to meet the requirements laid out in the business operations specifications and to create the internal efficiencies he identified, Tom proposes a design that includes three Web services to facilitate the product shipping business collaboration:

• ShippingService Web service. This is the supplier's Web service that is used to send and receive the details of the shipment pickup.
• PickupService Web service. This is the supplier's Web service that is used internally to be notified of product pickup and to confirm the shipment was picked up.
• TransportService Web service. This is the transport consolidator's Web service that is used by the supplier to initially order the transport and finally to confirm that the shipment was picked up.

Obviously, the TransportService Web service is not something that Northern Electronics can implement, because it belongs to the transport consolidator. But it is a needed part of Tom's design.

Tom works with Zack and Pam to coordinate the development of the TransportService Web service with the chosen transport consolidator business partner, Acme Consolidation Company. If the approach proves successful, Pam and her team can organize similar coordination with other transport consolidation business partners.

Note   The coordination with Acme Consolidation Company is outside the scope of this discussion, which simply relies on the coordination happening successfully.

Tom uses the business operations specifications to define the actual interfaces of the Web services. A more detailed description of the design process he uses is provided later in this document.

Web Service Health Model: Designing for Operations

The proposed three Web services allow messages to be sent and received in a systematic and verifiable manner. In addition to satisfying the business operations interface requirements, Tom also needs to think about how service instrumentation can be used to provide visibility and control over running instances of Web services. Web service instrumentation may take the following forms:

• Debugging traces can help with diagnosing code execution behaviors.
• Events are used to raise alerts that signify meeting certain management conditions.
• Performance counters are used to generate statistics that reflect the health of the services.
• Probes are internal service states that can be queried and controlled by management applications.

Tom needs to make sure the two Web services that Northern Electronics will implement can be designed for operations. This means that the Web services need to be designed to be managed to meet their performance objectives at multiple levels.

The IT organization uses a health modeling approach to develop a thorough understanding of how the Web services should perform, how to learn about how they are performing, and what to do when they are not performing according to plan.

Health modeling is a process within the design phase of the service life cycle to formally document the understanding of the system designers about what a healthy system is and what an unhealthy system is. It considers in advance all the manageable conditions the designers can anticipate. The health model that results contains detailed consideration of each anticipated manageable condition by laying out how to identify it and the actions to take to manage it. The health model details how to:

• Detect a manageable condition.
• Verify that it really is the condition.
• Diagnose what might be causing the condition as needed.
• Resolve the condition through corrective actions as needed.
• Re-verify the condition to see whether the system is in good health.

Tom works with others in the IT organization and relies on existing policies and procedures to formulate an initial health model for the two Web services he has proposed. This initial plan takes into account the company's standard system-level and application-level health models for Web services (for example, the requirement to perform "heartbeat" checking as a way to ensure a Web service is still running). Tom also needs to take into account the requirements in the business operations specifications.

More detailed information on Web service health modeling can be found in the Architecture Chronicles on Dynamic Modeling: Aligning Business and IT at the Microsoft Architecture Resource Center.

Product Shipping Web Services Solution Design

At a high level, Tom designs the flow of interactions between the individual workstations, Web services, and databases at Northern Electronics as shown in Figure 2. Although this document is primarily focused on the design concerns at Northern Electronics, it also describes the solution flow into the systems at Acme Consolidation Company to provide an end-to-end view of the automated process.

Figure 2. Product shipping service interaction flow

The following walkthrough begins with the assumption that the goods are already ordered by Wingtip Toys. The interaction flow outlines the automated process of obtaining transportation that will transport the goods to Wingtip Toys:

1. Generate order. Richard, the shipping clerk at Northern Electronics uses a transport ordering application to query the Shipping database to find POs that do not already have arranged transportation. Every PO has a status field that is used to indicate the state of processing. The possible states are:
   • Received. When a PO is received by the supplier.
   • Processed. When a PO has an associated transport order that is in ordered status.
   • Shipped. When the goods are loaded and transferred to the contracted trucker.
   • Completed. When the goods have been delivered and payment received.

   When a new PO is created and saved in the Shipping database, its status is set to Received and a new PO number is automatically generated. Therefore the Received status is used to search for POs that do not already have transportation ordered.

   For each resulting PO of the previous query, Richard generates a new transport order request that is saved in the Shipping database. Only the shipping clerk or shipping manager can create a transport order. Every transport order has one of the following statuses:

   • Initiated. When the supplier submits a transportation order.
   • Ordered. When the supplier has processed the transport order notification.
   • Arrived. When the truck has arrived at the supplier's warehouse.
   • Loaded. When the goods are loaded and transferred to the contracted trucker.
   • Completed. When the goods have been delivered to the wholesaler.

   When a new transport order request is saved in the Shipping database, its status is set to Initiated and a new transport order number is generated for the request.

2. Order transport. After the transport order is saved in the Shipping database, the transport ordering application creates a copy of the TransportOrder document with the order details and makes a Web service call to the transport consolidator's TransportService Web service.
3. Enter order request. The consolidator's TransportService Web service receives the transport order request and saves it in the Transport database.
4. Process order request. The transport consolidator clerk, Julie, uses a transport order processing application on the transport office workstation to query for new transport orders from the customers. For each new transport order, Julie finds a suitable trucking company to help fulfill the order. For example, after making a few phone calls to confirm with the trucking company, Julie enters the details in the transport order processing application to reflect that Blue Yonder Truckers will arrive on the following Tuesday at 10:00 A.M.
5. Pickup notification. When the transportation details are finalized, Julie uses the transport order processing application to create a PickupNotification document and sends it to the ShippingService Web service hosted by Northern Electronics.
6. Update pickup information. The ShippingService Web service receives the PickupNotification and verifies that the information complies with the specified delivery options (such as pickup time and date). If the conditions are met, the ShippingService Web service sends an acceptance of the notification to Acme Consolidation Company and updates the Shipping database with the arranged transport information. The transport order status is set to Ordered and the PO status is set to Processed. If Northern Electronics does not accept the PickupNotification, the acknowledgement reflects this and also explains why the notification is not okay. One reason for rejecting the notification might be because the pickup time specified is different from the requested time.
7. Query pickup. David, the loading dock clerk, uses a warehouse management application on the warehouse workstation to identify the goods that are going outbound to prepare the loading dock for load pickup. The application uses the internal PickupService Web service to query the Shipping database for pickup information. The PickupService Web service gets the list of transport orders that have the status set to Ordered, their respective POs, and prepares and returns a list of ItemsToPickup. The list is sorted according to the earliest time of pickup so that the goods can be prepared and queued in the right order on the loading dock. Based on the information in the transport order and POs, David arranges for the packaging of the items to be shipped and begins to fill out the appropriate documentation. This includes declaring the correct product identification and piece count. Northern Electronics must also declare that the parts they produce are not perishable, hazardous, or radioactive, and that the electronic parts do not include any batteries.
8. Confirm pickup. On the day of the pickup, Bob, the trucker, checks in with Richard. Richard checks Bob's identification and verifies his paperwork. Richard enters Bob's time of arrival into the system and looks up which dock Bob should go to. At this point, the transport order status will be updated to Arrived. Upon completion of loading, David uses the warehouse management application to enter the weight of the load in the transportation documentation. Bob enters his signature in a signatory device connected to the loading dock workstation to assume accidental liabilities during transportation of the goods. David uses the warehouse management application to confirm that the goods were picked up. The PickupService Web service updates the transport order status to Loaded and the corresponding PO status to Shipped in the Shipping database. This formally transfers liability of the goods from Northern Electronics to Acme Consolidation Company.
9. Pickup confirmation. The warehouse management application sends a pickup confirmation to the TransportService Web service to inform Acme Consolidation Company that the goods have been delivered to the trucker.

Designing the Web Services Message Contracts

One of the key implementation considerations is to ensure that the Web services on both sides will be able to communicate without having to agree on the vendor technology that is used to implement the services. For instance, Northern Electronics has chosen to implement their Web service clients and services using Microsoft .NET technology. However, Acme Consolidation Company should not be required to implement their Web services using the same Microsoft technology for the two companies to communicate.

To realize this goal, Northern Electronics and Acme Consolidation Company must first agree on the format and type of the messages that the Web services will send and receive so that they can communicate. To do this, the messages are described using industry standards such as XML Schemas (also known as XSD). Furthermore, it is also expected that the message schema description can be understood and consumed by the current generation of development tools to generate the corresponding software data structures for the respective development environment. For instance, XML Schema is one of the industry standard mechanisms for describing data formats and the Xsd.exe tool in Microsoft Visual Studio .NET can import a file containing the XML Schema representation of a business document to generate a Visual C# .NET class that the developers then use for implementing the Web services.

Table 1 and Table 2 provide a high-level description of the business documents that Acme Consolidation Company and Northern Electronics have agreed to exchange. The TransportOrder document is the business document that is sent to the transport service at Acme Consolidation Company. The high-level schema of this document is described in Table 1. The PickupNotification document is the business document sent by the Acme Consolidation Company to the ShippingService Web service. The document is described by the high-level schema in Table 2. The TimeWindow data type is used by the TransportOrder and PickupNotification documents and is described by the high-level schema in Table 3.

Table 1. TransportOrder Document High-Level Schema

Table 2. PickupNotification Document High-Level Schema

Table 3. TimeWindow Data Type High-Level Schema

For Acme Consolidation Company to invoke the ShippingService Web service, Northern Electronics must provide a service description for the Web service through a Web Services Description Language (WSDL) file. Sam, the lead developer at Northern Electronics, completes the following actions to generate the WSDL file that defines the Shipping Web service:

• First, Sam uses the XML Schema editor in Visual Studio to create the .xsd files that describe the format of the TransportOrder and PickupNotification documents. A developer at Acme Consolidation Company reviews and accepts the formats.
• Next, Sam uses the Xsd.exe tool in Visual Studio to consume the .xsd files produced in step 1 and then generate the corresponding Visual C# .NET classes.
• After that, Sam uses the ASP.NET Web service project template to create the Shipping Web service.
• Sam can then add a Web method named NotifyPickup() to the Shipping Web service. He specifies the Visual C# class for PickupNotification as the method parameter. This is the Web method that Acme Consolidation Company calls in order to send the PickupNotification document.
• Finally, Sam builds the Visual Studio project to render the WSDL for the Shipping Web service.

The WSDL file for the Web service is then made available to Acme Consolidation Company. Acme Consolidation Company uses the WSDL file to generate the Web service client proxy that will invoke the Shipping Web service.

Handling Business Exceptions

The previous scenario walkthrough of the product shipping solution shows how the system works in the best case situation—when everything goes as planned. However, the automated solution must also handle other conditions when real-world events interfere with the desired process flow. For example, one of the significant business exceptions to handle is when the truck fails to arrive on time to pick up the goods. Figure 3 shows the components in the solution that detects the truck non-arrival exception. In the normal flow of events, after the trucker checks in with the shipping clerk at Northern Electronics, the transport order status is updated to Arrived. To detect for the truck non-arrival event, Sam implements a business exception monitoring service that periodically queries the Shipping database for the list of transport orders with the current status set to Ordered and where the stated truck arrival time windows have expired.

The business exception monitoring service raises a truck non-arrival event in the Windows event log for each of these exceptions. A Microsoft Operations Manager 2005 (MOM) management agent monitors for the event and reports the event to the MOM server. A set of MOM rules is configured to raise a management alert that will trigger an e-mail notification to be sent to the relevant business personnel to handle the exception.

More information about modeling and implementing this business exception can be found in the Architecture Chronicles on Dynamic Modeling: Aligning Business and IT at the Microsoft Architecture Resource Center.

Figure 3. Detecting truck non-arrival event

Because of the unpredictability of the real world, it is often complicated and ineffective to try to completely model and automate the entire exception handling process. This statement definitely rings true for the preceding business exception. Therefore, after Tom, Jackie, and Pam evaluate the cost and benefits of automating the truck non-arrival exception handling process, they decide that after the appropriate individuals have been alerted, the solution will default to a human interaction model to try to resolve the exception. Figure 4 shows what happens after the e-mail notification is sent.

Figure 4. Handling truck non-arrival exception

The manual exception handling step begins after the shipping clerk receives the e-mail notification that states the identifier of the transport order violating the service agreement. The clerk uses the transport ordering application to retrieve the transport order and calls the transport ordering service department at Acme Consolidation Company. At this point, a series of phone conversations take place between business personnel in Northern Electronics, Acme Consolidation Company, and Blue Yonder Truckers, as well as with the trucker en-route to Northern Electronics to determine the cause of the delay (for instance, severe weather or poor road conditions) and to negotiate a new arrival time for the truck. Eventually, the shipping clerk at Northern Electronics records the outcome of the phone conversations and updates the TransportOrder document with the new arrival information. The updated transport order record is stored in the Shipping database and a copy of the document is also sent to the Acme Consolidation Company's TransportService Web service so that both parties have the new agreement in their systems' records.

Conclusion

This document described the application solution for enabling the automated transportation ordering process. It showed how the individuals involved use the different software components in the solution to realize the normal operation and exception flows in the business process. It showed how the solution architect used model-driven design and a service orientation approach to develop a flexible solution that was interoperable with business partners in a platform-neutral way, making use of Web services described using WSDL to communicate via messages specified and described using XML Schema. For the truck non-arrival event, the solution architect implemented a monitoring service that watches for situations where trucks fail to arrive at the loading dock at the designated time. An important point to note is that the solution seeks to provide real-time notifications of business operations conditions that need attention without attempting to automate the rectification actions which can be unduly complicated and costly to implement.

Additional Resources

Service Orientation and Its Role in Your Connected System Strategy on the MSDN Web site