Migrating Applications that Use Messaging Technologies
Updated: January 22, 2014
Authors: Kun Cheng
Contributors: Sreedhar Pelluru, Valery Mizonov, Christian Martinez, Rama Ramani
Reviewers: Steve Howard, Seth Manheim
Message queues are commonly used for applications to communicate with each other, or communicate within components of an application itself. In architecting for modern applications, architects and developers can use message queues to establish an asynchronous communication channel. A message queue allows senders to send messages and proceed with other tasks without waiting for a response from receivers. Receivers receive and process messages independently without blocking senders. The mechanism helps decouple otherwise tightly integrated components of an application and makes possible a more flexible and scalable solution.
When migrating an application from on-premises to Azure, we recommend that architects and developers examine the current architecture and identify possibilities of using Azure queues or Microsoft Azure Service Bus to take advantage of a loosely coupled architecture and the ability to scale, especially on the Azure platform.
Azure queues are based on Azure storage and provide a basic queuing mechanism to support point-to-point communication. Azure queues support access via REST-based HTTP or HTTPS. Each message queue supports a capacity of up to 100 TB (limit of current storage account). Each message can be up to 64 KB in size. See this article for more details.
The most common use of Azure queues is in a web role as a sender, to enqueue work items. You can also use them in a worker role as a receiver, to dequeue and process the work items asynchronously.
The Azure platform also offers queue-based messaging via Service Bus. In addition to queuing, Service Bus also provides secure messaging and relay capabilities that support distributed applications on Azure or hybrid deployments of applications across on-premises and Azure. For messaging mechanisms, Service Bus supports both point-to-point communication via Service Bus queues and publish-subscribe (pub-sub) via Service Bus topics and subscriptions. Service Bus topics and subscriptions allow multiple subscribers to listen to a single publisher at the same time. Relay capabilities enable hybrid solution scenarios in which enterprise assets on-premises or in private clouds can be extended, and communicate with cloud resources. Service Bus supports access via REST-based HTTP/HTTPS or the TCP protocol. Each Service Bus queue can be up to 5 GB in capacity. Each message can be up to 256 KB.
There are many differences between Azure queues and Service Bus queues. These differences include authentication, transaction support, and WCF integration. See Azure Queues and Azure Service Queues – Compared and Contrasted article for a detailed comparison between the two queuing technologies.
Windows applications commonly use Microsoft Message Queuing (MSMQ) as a queuing mechanism. MSMQ enables applications that run on separate servers in separate processes to communicate with each other in a durable, loosely coupled fashion. It also enables applications that reside in heterogeneous network environments to exchange information even when they are not online at the same time. It provides guaranteed message delivery, distributed transaction support, efficient routing, security, and priority-based messaging.
When migrating applications that rely on MSMQ technology to the Azure Platform, keep in mind that Azure does not currently support MSMQ. The migration requires you to change your code to use Azure queues. The following sections provide different options for migrating your applications that rely on MSMQ technology to the Azure platform.
Azure Service Bus
In many ways, Service Bus is the feature in Azure that most closely resembles MSMQ. They share similar functions such as basic queuing operations, transaction support, and dead lettering. However, using Service Bus requires different APIs than MSMQ, and the semantics differ in many ways. The following list provides a few key differences in terms of sizing and performance:
Service Bus message size is capped at 256 KB (both header and body), while MSMQ messages can be up to 4 MB in size.
Service Bus queues are limited in size to a maximum of 5 GB. MSMQ queue size is limited by computer hardware or configurable quotas.
Service Bus queue throughput can reach up to 2,000 messages per second. MSMQ throughput can reach more than 6,000 messages per second (based on a 1k benchmark). See Optimizing Performance in a Microsoft Message Queue Server Environment for more details. .
To facilitate the migration, it is possible to establish a link between MSMQ on-premises and Service Bus on Azure via a bridge. See the sample code here.
Azure queues provide a basic point-to-point communication channel. Unlike MSMQ, Azure queues do not support heterogeneous environments. In addition, they do not natively support features that a typical MSMQ environment does, such as automatic dead lettering, transactions, and ordering guarantees. However, application developers can implement the necessary features on top of Azure queues in order to achieve functionality similar to MSMQ. However, achieving this functionality does require application customization.
Azure Worker Role
With built-in support of MSMQ on Windows Server, running MSMQ in the same node as a worker role has the full functionality of MSMQ on-premises. However the worker role is subject to failover and service maintenance. When this happens, all the state information such as messages stored in the local MSMQ storage is lost and becomes unrecoverable. Unless MSMQ was used in a stateless fashion and the application is designed to be able to handle the role failover situation, it is not recommended to run MSMQ in a worker role instance.