1 out of 3 rated this helpful - Rate this topic

Integrating On-Premises Web Services with Windows Azure Service Bus and Port Bridge

Author: Tejaswi Redkar, Windows Azure Architect, Microsoft Services, Worldwide Services Community Lead--Windows Azure

Special Thanks to: Nagendra Mishr (Microsoft FAST Services Team) and Anna Skobodzinski (Architect, Microsoft Technology Center, NYC)

Last updated: September 14, 2011

In Integrating On-premises Search Engines with Windows Azure Web Applications I went over a real-world scenario where we integrated on-premises search engines with a web application running in Windows Azure. Immediately after that project, I had the opportunity to work on another project for integrating FAST Search with an application running in Windows Azure. The scenario was a bit different because this time the search engine was behind the firewall along with other line of business services. FAST is a very popular enterprise search engine used by a lot of large organizations for internal and external search. In this project, the customer already has a software-as-a-service (SaaS) application, but currently running in their own datacenter. They are currently evaluating the Windows Azure platform for elasticity and quicker time-to-market. The fun part in this project was that we used Windows Azure Service Bus to integrate with all of their on-premises services, including FAST, without writing a single line of code. In this note, I have shown how to accomplish this task.

Solution

In this project, the customer wanted to evaluate how their service will work in Windows Azure given that there is FAST-powered search and other shared services that will need to reside on-premises. These shared services are used by more than one application across multiple business units. Figure 1 below depicts the current application architecture.

Figure 1: Current Architecture

The current architecture is a multi-tiered architecture with RIA, Application, Common and Search services running in the same datacenter. The green blocks indicate that these components can be migrated to Windows Azure directly, whereas the red blocks indicate that these services need to remain on-premises because of dependencies or because they simply are not supported on Windows Azure (e.g. FAST). The application is designed for multiple tenants and the expected database growth might reach more than 300GB in the next 3 years. Therefore, during the evaluation phase, we had a dedicated session on architecting for the upcoming SQL Azure Federation features. You can find a list of SQL Azure Federation blogs and tools here. I won’t cover SQL Azure Federations in this note--maybe sometime later when it’s available in CTP.

The goal of the evaluation was to prove out whether we can:

Migrate all the application specific components (green boxes) to the Windows Azure platform
Integrate efficiently with the existing on-premises components

Implementation of Solution

Migration of the green boxes did not pose any issues. We were able to migrate all of those in a week’s timeframe. All eyes were focused on integration with shared services running in the customer’s datacenter. Because of the time constraints, we opted to modify and use the Port Bridge server and agent to provide seamless integration.

A note on Port Bridge

Port Bridge is a handy utility service created by Clemens Vasters. Port Bridge channels protocols over the Windows Azure Service Bus using the NetTcpRelayBinding. Typically, for integrating existing web services with Service Bus, you have to write custom WCF interfaces that wrap the Service Bus calls as a façade or wrappers. Port Bridge provides you with a generic wrapper service for channeling any protocol over Service Bus. I recommend you read the Port Bridge blog post on Clemens Vasters's blog before reading this article further.

Figure 2 below illustrates how we used Port Bridge to integrate the customer’s on-premises services.

Figure 2: Port Bridge and On-premises Integration

The communication steps in Figure 2 are as follows:

Port Bridge server was configured to register the FAST Query Server (15001), Common Services (80), and Authentication (443 for https) ports and hostnames in Service Bus. Note that the Port Bridge server can run on its own dedicated machine and you can also run multiple Port Bridge servers as long as they register unique endpoints within a namespace. The Port Bridge creates the endpoint URI on the fly by appending the service hostname to the service port. In Figure 2, the three services were running on three different machines and we had a dedicated Port Bridge server for FAST.
We ported the Port Bridge agent to a Windows Azure worker role and configured it to expose appropriate endpoints (internal/external) for the Search and Application services running in Windows Azure web roles to consume.
In the Search and Application services, we had to just modify the IP address and/or port numbers of the on-premises services to point to the Port Bridge agent worker role.

Thus, we could transparently invoke methods on all the on-premises services by just changing the configuration files. The final proof of concept architecture is illustrated in Figure 3 below.

Figure 3: Proof-of-Concept Architecture

Because there are so many hops in communications between on-premises and Windows Azure services, we did find latency as an issue here especially in search and some application services. We did not address these issues in the POC, but listed some follow-up investigation steps:

Try Windows Azure Connect instead of Service Bus
Leverage Windows Azure Caching wherever possible
Expose these services on the DMZ with secured endpoints for direct communications

Note
This architecture can definitely be improved upon for building a generic integration framework.

The Port Bridge server host mappings in app.config look like this:


<hostMappings>
   <add targetHost="commonserviceshost" allowedPorts="80"  />
   <add targetHost="authservicehost" allowedPorts="443"  />
   <add targetHost="fastserverhost" allowedPorts="15001"  />
</hostMappings>

In the host mapping, there are three services representing the Common Services, Authentication Service, and FAST Server.

Next, on the Port Bridge agent, the port mapping configuration looks like this:


<port localTcpPort="80" targetHost="commonserviceshost" remoteTcpPort="80">
   <firewallRules>
      <rule source="127.0.0.1"/>
      <rule sourceRangeBegin="98.0.0.0" sourceRangeEnd="98.255.255.255"/>
      <rule sourceRangeBegin="65.0.0.0" sourceRangeEnd="65.255.255.255"/>
      <rule sourceRangeBegin="70.0.0.0" sourceRangeEnd="70.255.255.255"/>
      <rule sourceRangeBegin="131.0.0.0" sourceRangeEnd="131.255.255.255"/>
      <rule sourceRangeBegin="10.0.0.0" sourceRangeEnd="10.255.255.255"/>
      <rule sourceRangeBegin="24.0.0.0" sourceRangeEnd="24.255.255.255"/>
   </firewallRules>
</port>
<port localTcpPort="443" targetHost="authservicehost" remoteTcpPort="443">
   <firewallRules>
      <rule source="127.0.0.1"/>
      <rule sourceRangeBegin="65.0.0.0" sourceRangeEnd="65.255.255.255"/>
      <rule sourceRangeBegin="98.0.0.0" sourceRangeEnd="98.255.255.255"/>
      <rule sourceRangeBegin="70.0.0.0" sourceRangeEnd="70.255.255.255"/>
      <rule sourceRangeBegin="131.0.0.0" sourceRangeEnd="131.255.255.255"/>
      <rule sourceRangeBegin="10.0.0.0" sourceRangeEnd="10.255.255.255"/>
      <rule sourceRangeBegin="24.0.0.0" sourceRangeEnd="24.255.255.255"/>
   </firewallRules>
</port>
<port localTcpPort="15001" targetHost="fastserverhost" remoteTcpPort="15001">
   <firewallRules>
      <rule source="127.0.0.1"/>
      <rule sourceRangeBegin="65.0.0.0" sourceRangeEnd="65.255.255.255"/>
      <rule sourceRangeBegin="98.0.0.0" sourceRangeEnd="98.255.255.255"/>
      <rule sourceRangeBegin="70.0.0.0" sourceRangeEnd="70.255.255.255"/>
      <rule sourceRangeBegin="131.0.0.0" sourceRangeEnd="131.255.255.255"/>
      <rule sourceRangeBegin="10.0.0.0" sourceRangeEnd="10.255.255.255"/>
      <rule sourceRangeBegin="24.0.0.0" sourceRangeEnd="24.255.255.255"/>
   </firewallRules>
</port>

The values of the targetHost and the remoteTcpPort and allowedPorts attributes must match between the Port Bridge server and the Port Bridge agent for the Service Bus to resolve to the appropriate endpoint.

The firewall rules represent an extra layer of security for communicating with the Port Bridge agent. Because in this case the Port Bridge agent was running in a Windows Azure role, we had to open the Windows Azure IP addresses. You can download the Port Bridge source code from Clemens Vasters's blog here: http://vasters.com/clemensv/2009/11/18/Port+Bridge.aspx.

Warning
Port Bridge is not production quality source code and is not supported by Microsoft. Please review the source code yourself and run your testing tools against it before running it in a production environment.

You can now try running the scenario yourself. The best way to start is to:

First create a simple WCF web service running on-premises.
Create a console and a web client for this WCF service.
Test the connectivity between the client and the WCF server.
Run Port Bridge server to register the endpoint of this WCF service.
Run the Port Bridge agent from the source code on another machine on-premises mapped to the registered endpoint.
Run the client on the same machine as the Port Bridge agent but this time change the configuration file of the client app to point to the Port Bridge agent’s mapped port instead of the WCF service directly.
Test the results. The method call should go from Client-->Port Bridge Agent-->Service Bus-->Port Bridge Server-->WCF service.
Next, create a worker role and add the Port Bridge agent code to the Run() method of the worker role; alternatively start PortBridgeAgent.exe from the Run() method or startup task. I had simply copied the Port Bridge agent code to the Run() method.
Open the appropriate endpoints (internal or external) to communicate with the Port Bridge agent on the mapped ports.
Add the web application client you created in step 2 to the cloud service project and, similar to step 6, point the web application to the Port Bridge agent worker role’s endpoint (external or internal) instead of the WCF endpoint directly.
Deploy the cloud service.
Run the web application and call the web service, the call should traverse the following path.

Web App-->Port Bridge Agent Worker-->Service Bus-->Port Bridge Server-->WCF service

Note that you don’t have to modify the Port Bridge Server or the WCF service running on-premises at all.

Below is the code snippet for running the Port Bridge agent as a Windows Azure worker role. The StartPortPridgeAgent() function is called once, in the worker role’s Run() method.

private void StartPortBridgeAgent()
{
   try{
      Trace.WriteLine("Starting PortBridgeAgent.");
      PortBridgeAgentSection settings =
      ConfigurationManager.GetSection("portBridgeAgent") as 
      PortBridgeAgentSection;
      if (settings !=  null)
      {
         serviceNamespace = settings.ServiceNamespace;
         issuerName = settings.IssuerName;
         issuerSecret = settings.IssuerSecret;
      }
      Trace.WriteLine("PortBridge Agent Settings acquired");
      host = new PortBridgeClientForwarderHost();
      if (settings !=  null && settings.PortMappings.Count > 0)
      {
         int i = 1;
         foreach (PortMappingElement mapping in settings.PortMappings)
         {
            List<IPRange> firewallRules = new List<IPRange>();
            if (mapping.FirewallRules != null && mapping.FirewallRules.Count > 0)
            {
               foreach (FirewallRuleElement  rule in mapping.FirewallRules)
               {
                  if (!string.IsNullOrEmpty(rule.SourceRangeBegin)  &&
                  !string.IsNullOrEmpty(rule.SourceRangeEnd))
                  {
                     firewallRules.Add(newIPRange(IPAddress.Parse(rule.SourceRangeBegin), 
                     IPAddress.Parse(rule.SourceRangeEnd)));
                  }
                  else if (!string.IsNullOrEmpty(rule.Source))
                  {
                     firewallRules.Add(newIPRange(IPAddress.Parse(rule.Source)));
                  }
               }
            }
            if (mapping.LocalTcpPort.HasValue)
            {
               if (!string.IsNullOrEmpty(mapping.LocalPipe)  ||
               !string.IsNullOrEmpty(mapping.RemotePipe))
               {
                  throw newConfigurationErrorsException(string.Format(
                  "LocalTcpPort  {0} defined with incompatible other settings",
                  mapping.LocalTcpPort.Value));
               }
               elseif  (!mapping.RemoteTcpPort.HasValue)
               {
                  thrownew ConfigurationErrorsException(string.Format(
                  "LocalTcpPort  {0} does not have a matching RemoteTcpPort defined",
                  mapping.LocalTcpPort.Value));
               }
               IPEndPoint endPoint =
               RoleEnvironment.CurrentRoleInstance.InstanceEndpoints
               [string.Format("TcpEndpoint{0}",  i++)].IPEndpoint;
               Trace.WriteLine("IPEndpoint for  PortBridge Agent: 
               " + endPoint.ToString());
               host.Forwarders.Add(newTcpClientConnectionForwarder
               (serviceNamespace, issuerName, issuerSecret, mapping.TargetHost,
               mapping.LocalTcpPort.Value, mapping.RemoteTcpPort.Value, 
               mapping.BindTo, useHybrid, firewallRules, endPoint));
               //}
               Trace.WriteLine("Created  TCP Connection");
            }
            if (!string.IsNullOrEmpty(mapping.LocalPipe))
            {
               if (mapping.LocalTcpPort.HasValue ||
               mapping.RemoteTcpPort.HasValue)
               {
                  thrownewConfigurationErrorsException(string.Format(
                  "LocalPipe  {0} defined with incompatible other settings",
                  mapping.LocalPipe));
               }
               elseif (string.IsNullOrEmpty(mapping.RemotePipe))
               {
                  thrownewConfigurationErrorsException(string.Format(
                  "LocalPipe  {0} does not have a matching RemotePipe defined",
                  mapping.LocalPipe));
               }
               host.Forwarders.Add(newNamedPipeClientConnectionForwarder
               (serviceNamespace,  issuerName, issuerSecret, mapping.TargetHost,
               mapping.LocalPipe,  mapping.RemotePipe, useHybrid));
            }
         }
      }
      else
      {
         List<IPRange>  firewallRules = new List<IPRange>();
         firewallRules.Add(newIPRange(IPAddress.Loopback));
         host.Forwarders.Add(newTcpClientConnectionForwarder
         (serviceNamespace,  issuerName, issuerSecret, cmdlineTargetHost, 
         fromPort, toPort, null, useHybrid, firewallRules, null));
      }
      Trace.WriteLine("Opening  Host.");
      host.Open();
      Trace.WriteLine("Opened  Host.");
   }
   catch (Exception  ex)
   {
      Trace.WriteLine("Error in StartPortBridgeAgent() " + ex.Message);
      throw ex;
   }
}

References

Port Bridge by Clemens Vasters

Did you find this helpful? Yes No

Not accurate

Not enough depth

Need more code examples

(1500 characters remaining)