Service Station: WSE 3.0, SOAP Transports, and More

Magazine > Issues > 2006 > June > Service Station: WSE 3.0, SOAP Transports, and ...

Service Station

WSE 3.0, SOAP Transports, and More

Aaron Skonnard

Code download available at: ServiceStation2006_06.exe (175 KB)
Browse the Code Online

It's that time again. Time to answer some of the questions I get on a regular basis. This month I'll look at service orientation and policy-based compatibility, SOAP's transport-neutral design, and Web Services Enhancements (WSE) 3.0.

Q I know SOAP was designed as a transport-neutral messaging framework. Is it really possible to design WSE/Windows® Communication Foundation applications without taking into account transport specifics?

I know SOAP was designed as a transport-neutral messaging framework. Is it really possible to design WSE/Windows® Communication Foundation applications without taking into account transport specifics?

A SOAP was indeed designed as a transport-neutral XML messaging framework. One of its primary goals was to enable the packaging of protocol information within XML messages so that such information would not have to be carried at the transport level. This allows XML protocols (such as those defined by WS-* in the areas of security, reliable messaging, and transactions) to be used across a variety of transport communication scenarios.

A typical SOAP stack provides a few different layers of code (Figure 1 shows an example). The topmost layer is the application code. This is either the client application that consumes a service or the service business logic itself, depending on which side of the wire you're talking about. (Most stacks provide a programming model that makes it easy to program and consume services.) Then there's the SOAP messaging layer. You can think of this layer as the infrastructure that implements SOAP and the various WS-* protocols. This code processes the various SOAP headers using interception techniques—the process is completely hidden from you at the application layer. (Most stacks also provide a layer that abstracts away transport specifics, shielding the layers above from such communication details.) Finally, there's the physical network where bytes meet the wire. This layering model holds true for both clients and services.

Figure 1 SOAP Stack

With the layering shown in Figure 1, the code you write at the application layer is completely shielded from the underlying transport details. Given this design, you shouldn't have to modify the application code in order to use a different transport, as long as the new transport is compatible with your service requirements. Of course, the feasibility of such flexibility depends largely on the implementation stack and the issues surrounding "compatibility."

WSE 2.0 was the first Microsoft product to bring SOAP's promise of transport-neutral XML messaging to life when it introduced support for TCP as an alternative to HTTP. WSE 2.0 relied on a layered approach similar to the one shown in Figure 1. It allowed you to write services that could run over either TCP or HTTP without changes to the service logic. WSE 3.0 builds on the WSE 2.0 foundation, making a few changes to the topmost layers in order to simplify the developer experience.

The Microsoft next-generation Web services platform, Windows Communication Foundation, was designed according to the general architecture shown in Figure 1. It too allows a single service to run over multiple transports, at the same time even, and it supports more protocols than just HTTP and TCP (MSMQ and P2P, for instance). Windows Communication Foundation also provides a more sophisticated SOAP messaging layer, which supports more WS-* specifications. With all of these stacks, developers can write custom transports and plug them in, providing truly unlimited transport possibilities.

Now, let's get back to your question. Is it really feasible to write WSE/Windows Communication Foundation services without worrying about transport specifics so administrators can choose the transports at run time? It's plausible, but achieving success with such an approach is highly doubtful. (It's probably as feasible as the promise that ODBC will allow administrators to swap out an application's underlying database after development.)

When designing services, there are numerous issues related to the communication transport that must be considered earlier rather than later. For example, if your service code requires request-response semantics and, therefore, a two-way transport, an administrator can't configure it to use the Microsoft® Message Queue (MSMQ) transport, which is inherently one-way. Or imagine a transport developed for a special channel that provides numerous optimizations but also requires all messages to be smaller than 10KB in size. An administrator might then configure a service with this special transport (to speed things up). While this configuration might pass initial QA tests, it would almost certainly fail down the road once it encounters messages that exceed the size limit. Only the developer can ensure success with the special transport by making the appropriate design considerations up front. In general, administrators will only be able to use transports that are completely compatible with your service design/implementation.

Developers simply cannot ignore transport details when designing services. Likewise, administrators cannot ignore the service design when choosing a transport. In fact, developers should be able to impose requirements that influence the type of transport used by administrators at run time. This would help administrators make better choices. Windows Communication Foundation helps facilitate this via special attributes that developers can apply to their code (the [DeliveryRequirements] attribute, for example). At run time, Windows Communication Foundation validates that the delivery requirements (specified by the developer) match the actual service configuration during startup. Developers can even write custom Windows Communication Foundation validators. These techniques are helpful because they encourage communication between developers and administrators.

Despite this reality, separating the SOAP programming model from the underlying transport layer is still a good thing. It's helpful to both developers and administrators. It makes everyone's life a bit easier. Think of it as a way to increase productivity and flexibility. One could argue that this particular SOAP design goal has been realized to as great a degree as possible.

Q One of the tenets of service orientation is "policy-based compatibility." This seems to suggest that policies should be used to negotiate behavior. Is that actually the case with WSE or Windows Communication Foundation?

A A policy is an ordered set of assertions that describe a service's requirements or capabilities. Policies describe semantic requirements or capabilities that cannot be expressed in structural contract languages like XML Schema Definition (XSD) and Web Services Description Language (WSDL). For example, a particular security policy may include assertions that specify the type of token required for client authentication as well as integrity and privacy requirements for each message exchange. The assertions might specify exactly what portions of each message need to be signed and/or encrypted along with what key to use for each task.

The tenet you refer to states that service compatibility should be determined based on policy information. This implies that policy information must be shared between services and clients, which may be implemented using different stacks. In order for this to work, the industry needs a common framework for expressing policy in a platform-neutral manner. WS-Policy exists for this purpose.

In addition to a common policy framework, standard policy assertions are also necessary to negotiate compatibility and protocols in specific areas. WS-PolicyAssertions and WS-SecurityPolicy are examples of specifications that define standard policy assertions that all Web service frameworks can agree on. These specifications define the structure of each policy assertion along with its meaning. Similar specifications that focus on different functional areas will surely appear over time.

In theory, a service could publish a policy containing predefined assertions and somehow make the policy available to interested clients. A client would then use the service policy to determine whether it is capable of meeting the service's requirements (based on its understanding of the assertions). When the client is compatible it could even use the policy to automate its support for the specified requirements and protocols. The key is that code generators need to become policy-aware so they can automate the client experience when the service is using more than just the WS-I basic profile.

Although policy has long been touted as a necessary piece of the service-oriented puzzle, support for exchanging policies and policy-aware code generation hasn't materialized in today's stacks. Developers working with WSE 3.0, for example, must manually define the policy used within each application, even in clients. You may be able to manually reuse the same policy file within a service and a client (assuming you're using WSE 3.0 in both applications) but it requires a human to copy the file and to configure each application individually. Today's code generators (wsdl.exe and wsewsdl3.exe) don't look at policy information—they look only at XSD and WSDL to generate structure contracts.

In reality, WSE 3.0 uses policy as a configuration language and not as an exchange format for negotiating behavior. There is still value in this declarative approach since it decouples common messaging needs from the service logic itself. This separation gives developers the freedom to modify the policy (configuration) file to change service behavior without having to recompile.

The good news is that Windows Communication Foundation is poised to change this process. Windows Communication Foundation doesn't use policy as a configuration format; it defines a much simpler configuration language for developers to use (via the concept of bindings).

Windows Communication Foundation translates binding information into WS-Policy statements embedded within the service's WSDL definition. This makes it easy to share the complete service description with other parties. As long as consumers understand how to process the embedded WS-Policy statements, service compatibility can actually be negotiated (and even automated). When you generate a Windows Communication Foundation proxy using svcutil.exe, it automatically builds the appropriate client configuration based on the policy statements embedded within the WSDL or it informs you of any potential incompatibilities. It would be nice if similar support would come to fruition across non-Microsoft frameworks as well.

Q How will I port a WSE 3.0 filter to Windows Communication Foundation? Is the Windows Communication Foundation notion of binding equivalent to the WSE 3.0 notion of policy?

A WSE 3.0 makes use of a "pipeline" to implement the SOAP processing layer you saw back in Figure 1. The WSE pipeline consists of an ordered sequence of SoapFilter objects, each of which gets a chance to process the SOAP message en route. You can customize message processing by injecting your own SoapFilters into the pipeline. This is done via the WSE 3.0 policy framework. For example, you could inject a custom filter to handle validation, security, or logging needs.

Moving SoapFilters forward to Windows Communication Foundation will require some significant reworking. Windows Communication Foundation provides a user-friendly programming model similar to what's offered by ASMX. In Windows Communication Foundation parlance, we refer to this as the "service model" layer (because the root namespace is called System.ServiceModel). It also provides a sophisticated messaging layer and the underlying transport layers. Windows Communication Foundation provides two extensibility pipelines: one within the service model layer (on the dispatcher/proxy) and one within the messaging layer (the channel stack). This is a significant difference from the WSE 3.0 architecture.

When working with Windows Communication Foundation, you influence the channel stack by choosing (configuring) a binding. The channel stack is concerned with what goes on the wire, which requires agreement and coordination with the other side. Hence, Windows Communication Foundation translates binding information into policy statements (which show up in the auto-generated WSDL) so the other applications can build equivalent channel stacks in their environments. The policy is no longer just a configuration mechanism (like it was in WSE 3.0) but rather a product of the system. You can think of a binding as the Windows Communication Foundation configuration mechanism, which results in a policy when the endpoint description is shared via WSDL.

You configure the service model dispatcher/proxy using contracts. The dispatcher/proxy also has an associated pipeline of "behaviors" that influence local execution details. You configure behaviors using either custom attributes in your code or custom elements in your configuration file.

When moving WSE 3.0 SoapFilter code forward to Windows Communication Foundation, you must first decide whether the functionality belongs within the channel stack or the dispatcher/proxy. This largely depends on whether it requires coordination with the other side. If your SoapFilter requires some level of agreement between the client and the service (like security), you would implement the SoapFilter as a custom Windows Communication Foundation channel along with a custom binding element. You would use the custom binding element on your endpoint to inject the functionality. This allows your binding element to produce policy statements for inclusion in the generated WSDL. You can then configure svcutil.exe to recognize the policy statements and produce your binding element in the generated proxy configuration. This approach is required for any application-level protocol that requires coordination across the wire.

If your SoapFilter does not require coordination between both sides (as with validation or logging), you'd implement the SoapFilter as a Windows Communication Foundation behavior that inserts the appropriate interface in the proxy/dispatcher. Windows Communication Foundation provides hooks for many types of behaviors including parameter inspection, message formatting, message inspection, operation selection, and operation invocation. The difference is that these extensibility points only affect the local execution behavior of the Windows Communication Foundation stack.

Moving lower-level extensibility code almost always requires some significant reworking and this particular transition is no exception to the rule.

Q When using the WSE 3.0 turnkey security assertions, what must I do to integrate future Windows Communication Foundation applications?

A ASMX, WSE 3.0, and Windows Communication Foundation will all be compatible on the wire. ASMX and WSE 3.0 clients will be able to consume Windows Communication Foundation services as long these services don't use any WS-* protocols that aren't supported by ASMX or WSE. Likewise, Windows Communication Foundation clients will be able to consume ASMX and WSE 3.0 services. Obviously, Windows Communication Foundation provides more functionality and support for WS-* than its predecessors, which means you'll need to pay close attention to your configuration in order to achieve compatibility.

If you're trying to integrate Windows Communication Foundation with existing ASMX or WSE applications that conform to the basic profile, use BasicHttpBinding within Windows Communication Foundation. If, however, you're using message-based security via the WSE 3.0 turnkey assertions, you need to make sure you use a compatible binding configuration. For example, when using the UsernameOverTransport assertion in WSE 3.0, you should configure your Windows Communication Foundation binding in the following manner:

<basicHttpBinding>
   <binding name="UsernameOverTransportBinding">
      <security mode="TransportWithMessageCredential">
         <message clientCredentialType="UserName" />
      </security>
   </binding>
</basicHttpBinding>

If you happen to be using the CertificateMutualAuthenticationProfile, you should still use BasicHttpBinding, but instead configure it this way:

<basicHttpBinding>
   <binding name="CertificateMutualAuthenticationProfileBinding">
      <security mode="Message">    
         <message clientCredentialType="Certificate"
            negotiateServiceCredential="false"/>
      </security>    
   </binding>
</basicHttpBinding>

In Windows Communication Foundation, WSHttpBinding provides support for more sophisticated security scenarios. You need to use this when integrating with WSE 3.0 applications using the other turnkey security assertions—you have to customize the binding differently for each turnkey security assertion. For example, if you're using the UsernameOverCertificate assertion on the WSE 3.0 side, you must configure the WSHttpBinding binding, as shown here:

<wsHttpBinding>
   <binding name="UsernameOverCertificateBinding">
      <security mode="Message">
         <message clientCredentialType="UserName"         
                  negotiateServiceCredential="false"/>
      </security>   
   </binding>
</wsHttpBinding>

Microsoft will provide more guidance on integrating Windows Communication Foundation with existing ASMX and WSE 3.0 applications as it gets closer to shipping.

Q Is Microsoft planning to provide any tools for analyzing a security policy?

A Microsoft Research is working on a project, code-named "Samoa,"for building tools to analyze security policies. The goal of the project is to "exploit recent theoretical advances in the analysis of security protocols in the practical setting of XML Web services." The group has produced a tool called the Policy Advisor that works with both WSE 2.0 and WSE 3.0. In fact, it actually ships with WSE 3.0 as a sample (you'll find it in the default samples directory).

The Policy Advisor examines the configuration and policy files for one or more WSE endpoints and produces a human-readable report which highlights typical security risks and provides some basic advice. You can use this tool to review your existing WSE 3.0 applications without making any changes to your application code. The generated report provides valuable fodder for security reviews and related discussions.

The tool doesn't require any installation because it's simply an XSLT file that processes configuration files and WSE 3.0 policy cache files, looking for potential security issues to consider. In order to use the tool, you must first author a WSE endpoint file that defines a list of endpoints you'd like the tool to inspect. The code in Figure 2 illustrates the format of such a file.

Figure 2 WSE Endpoint File

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="http://PolicyAdvisor.xml" type="text/xsl"?>
<endpoints xmlns="http://schemas.microsoft.com/pa/2005/10/endpoints">
  <endpoint name="Stock Service client"
      path="StockServiceClient\" 
      config="app.config"/>

  <endpoint name="Stock Service service"
      path="StockService\" 
      config="Web.config"/>

  <endpoint name="SC client"
      path="SecureConversationPolicyClient\" 
      config="app.config"
      policyCache="wse3policyCache.config"/>
  
  <endpoint name="SC server"
      path="SecureConversationPolicyService\" 
      config="Web.config"
      policyCache="wse3policyCache.config"/>
  ...
</endpoints>

Notice that the file contains an xml-stylesheet processing instruction that associates the file with the PolicyAdvisor.xml XSLT file. Then it simply contains a list of <endpoint> elements, each of which defines the path to the project, the name of the configuration file, and the name of the WSE policy file.

In order to run the policy advisor, simply open the endpoint file in Internet Explorer®, which causes the XSLT processing to start. You'll see the report displayed in the browser (see Figure 3). If you'd like more information on the Samoa project, you should check out research.microsoft.com/projects/samoa.

Figure 3 Policy Advisor for WSE

Q How do you change aspects of a policy that are not exposed by the WSE 3.0 policy wizard?

A The WSE 3.0 security policy wizard provides step-by-step guidance for building a security policy with the new turnkey security policy assertions. Hence, the wizard only lets you make changes related to your security policy—it's not a generic policy editor. If you need to modify a policy that was built with the policy wizard and you need to add/change elements not related to the security assertions, you need to use an XML editor to work on the policy.

Luckily, the WSE 3.0 policy format is much simpler than WS-Policy (used in WSE 2.0) and isn't hard to work with directly. Plus, if you're working with policy files in Visual Studio® 2005, you'll notice helpful XML IntelliSense® that makes working directly with XML policies even easier.

Q Is it possible to use the WSE configurations tool without running Visual Studio?

A Yes, with WSE 3.0, you can now run the WSE configuration tool outside of Visual Studio 2005. You used to have to open the Visual Studio project and select WSE Settings from the project menu to launch the tool. This didn't allow you to modify configuration settings in deployment environments where Visual Studio wasn't installed.

Once installed, you can launch the tool from the start menu (Start | All Programs | Microsoft WSE 3.0 | Configuration Tool. The executable is named WseConfigEditor3.exe and it's found in the Tools subfolder of the installation directory. After launching the tool, you can choose either to create a new configuration file or open an existing one from the File menu (see Figure 4). After you've made your changes you can save the file.

Figure 4 WSE 3.0 Configuration Tool

Q If WSE 3.0 supports both HTTP and TCP, why doesn't the following line of code work?

SoapReceivers.Add("http://localhost:8080/math/", typeof(MathService));

Q If WSE 3.0 supports both HTTP and TCP, why doesn't the following line of code work?

SoapReceivers.Add("http://localhost:8080/math/", typeof(MathService));

A WSE 3.0 was designed to make use of the IIS/ASP.NET infrastructure for receiving HTTP messages. Most developers configure WSE 3.0 on existing ASMX implementations deployed in IIS. However, WSE also provides some messaging classes (SoapReceiver/SoapService) that implement IHttpHandler in order to integrate WSE services with IIS/ASP.NET without using ASMX. But in all cases, WSE requires the use of IIS/ASP.NET for receiving incoming HTTP messages.

If you need to host a WSE service within a non-IIS application (such as a Windows service or a rich client), you need to use the self-hosting technique I discussed earlier (SoapReceivers). But when using self-hosting, WSE 3.0 only supports the TCP transport. WSE can spin up a TCP listener within any application to wait for incoming messages, but the same is not true for HTTP since it relies on IIS.

The introduction of HTTP.SYS (with Windows XP SP2 and Windows Server™ 2003) makes it possible to support self-hosting over HTTP in future releases, and this is precisely what Windows Communication Foundation does. With Windows Communication Foundation, you can spin up an HTTP listener in any .NET-based application and wait for incoming HTTP messages. Since Windows Communication Foundation is already on the doorstep, it's unlikely that WSE will ever provide this feature. Nevertheless, if you really want this capability in WSE, the WSE 3.0 extensibility points make it possible to implement a custom HTTP.SYS transport on your own.

The .NET Framework 2.0 provides a System.Net.HttpListener class for easily integrating with the underlying HTTP.SYS driver. All you need to do is write the code to integrate the HttpListener class with the SOAP transport layer provided by WSE 3.0 (that is, you need to write a SoapTransport class). Once the custom transport is written, you configure your applications to use it (instead of the default HTTP transport tied to IIS/ASP.NET), as shown here:

<configuration>
  <microsoft.web.services3>
    <messaging>
      <transports>
        <add scheme="http" 
         type="Skonnard.Samples.HttpSys.HttpSysTransport, 
               Skonnard.Samples.HttpSys"/>
      </transports>
    </messaging>
  </microsoft.web.services3>
</configuration>

With this configuration in place, the line of code shown in your question will work (assuming, of course, you have a functional HTTP.SYS transport implementation).

I've provided a complete sample that illustrates how to implement a custom HTTP.SYS transport. This is nothing more than a sample and is by no means production ready, but it's a good starting point for those who need to investigate this approach further. You can download the complete sample from the MSDN®Magazine Web site.

Send your questions and comments for Aaron to sstation@microsoft.com.

Aaron Skonnard is a cofounder of Pluralsight, a Microsoft .NET training provider. Aaron is the author of Pluralsight's Applied Web Services 2.0, Applied BizTalk Server 2006, and Introducing Windows Communication Foundation courses. Aaron has spent years developing courses, speaking at conferences, and teaching professional developers. Reach him at pluralsight.com/aaron.