Interacting with Message Queues
Application Developers Training Company
Summary: Examines how to use the System.Messaging namespace in the Microsoft .NET Framework to implement Microsoft Message Queue in your applications. (19 printed pages)
- Understand the basics of Message Queuing
- Learn how the .NET Framework provides features in the System.Messaging namespace for message queuing
- Explore various common programming scenarios for using message queues in an application
The following should be true for you to get the most out of this document:
- You understand Microsoft® Windows® programming with COM/COM+ and DCOM, as well as the fundamentals of the .NET Framework in Microsoft® Visual Basic® .NET.
- You have designed and built distributed, disconnected applications, with or without inter-application messaging.
One of the most important Microsoft technologies for distributed applications is Message Queuing (MSMQ), originally released with the Windows NT 4.0 Option Pack. MSMQ provides an inter-application messaging infrastructure, meaning that it provides the features for sending messages between disconnected applications. This is a crucial requirement for using middle-tier components in an application.
MSMQ does a number of things for an application. It provides:
- Message Transport. Using COM and DCOM, MSMQ handles packaging the message and transporting it across the network to the receiving application. It uses its own IP-based protocol as well as HTTP for transport, and COM/DCOM for object serialization when the body includes an object.
- Resilient Queues. An MSMQ queue is an in-memory or persistent store of messages waiting for delivery. An application can "peek" at messages without removing them, filter them, and retrieve them from the queue.
- Patient Disconnection. Unless you specify otherwise, a message will sit and wait in a queue until an application comes looking for it. This means that MSMQ supports disconnected applications, where computers are not connected to the same physical network. Once the machines are reconnected, the messages are transmitted to the receiver applications.
- Transactional Messages. By using the optional transaction features of MSMQ, you can be guaranteed that a message is delivered once and only once; multiple messages are delivered in a particular order; and if anything goes wrong, the entire set of messages rolls back to its initial state. MSMQ is a standard resource manager, so it can coordinate activity with other resource managers, such as Microsoft® SQL Server, so messages and data can be read and written in the same transaction.
- Error Handling and Auditing. A lot can go wrong with any kind of disconnected application, and MSMQ has to provide robust support for error and audit conditions, such as messages that are undeliverable, and journaling messages sent and received for auditing purposes.
- Various APIs. MSMQ provides a set of COM interfaces with an extensive set of features for creating, transporting, and receiving messages that are easy to use from scripts and Visual Basic, and provides the infrastructure for Queued Components in Windows 2000's COM+.
Figure 1 shows an overall view of how MSMQ works between two applications. Using one of the programming interfaces provided by MSMQ, an application creates a message with some kind of data payload. MSMQ, through the Queue Manager, takes care of marshaling the data in the proper format for enqueuing the message. The Queue Manager also takes care of locating the destination queues (there may be more than one), whether they are on the local machine or on a remote machine over the network.
Figure 1. MSMQ provides the infrastructure to send and receive messages between applications
The Queue Manager is required on both sender and receiver machines to take full advantage of queuing. It manages the process, with support for locating a queue, publishing messages, creating and maintaining queues, and retrieving messages.
As simple conceptually as MSMQ seems, the original Win32 APIs were, and still are complex, hard to use, and difficult to make sense of—unless you relish low-level programming. A COM interface eases the burden somewhat. But in the .NET Framework, Microsoft has provided a namespace with a set of classes that support MSMQ, System.Messaging. Although MSMQ programming still isn't easy, the programming techniques are now consistent with other systems programming using the .NET Framework.
Note MSMQ sounds a lot like SOAP, the XML-based method invocation protocol. Although there is some overlap, SOAP is primarily a way to cause code to execute on a remote machine using cross-platform technologies. MSMQ uses COM for serialization and COM+ services, and provides transactional queues that SOAP doesn't address. MSMQ provides a reliable transport for messages that are expressed in SOAP, but SOAP is only one of many message formats.
System.Messaging is the namespace containing the set of classes that wrap the underlying MSMQ infrastructure. It consists of a few dozen classes.
Out of all these classes, there are three objects that you'll use most often when programming MSMQ: MessageQueue, Message, and MessageEnumerator. This paper will discuss these objects in some detail, and look at some code for using them in your Visual Basic .NET applications.
The MessageQueue class is the primary object for interacting with message queues on local or remote machines. Use it to enumerate queues on a particular machine, retrieve messages from a queue, send messages to a queue, create and delete queues, and so on. Most everything you do with MSMQ in the .NET Framework starts with an instance of the MessageQueue class.
As you might imagine, with a class as complex as MessageQueue, the class provides a complicated set of fields, properties, methods, and events. There are several categories of interface elements that you'll use to perform various tasks. The following sections introduce the various tasks you'll need to take on, when using the MessageQueue class.
The Create and Delete methods do just what their names say. Given a reference to a queue (you'll learn the ways to reference a queue later in this section), these methods create a new queue or delete an existing queue on the machine you specify. The methods throw the logical exceptions when you try to create a queue with the same name as an existing queue, when you delete a non-existent queue, or if you lack proper permissions to take the specified action.
The Exists method returns a Boolean value indicating whether the specified queue exists in the location you specify. The Purge method dumps all the existing messages in a queue without retrieving them.
A set of methods that retrieve values, all prefixed with the word "Get" retrieve a collection of queues in the location and of the type specified. These methods provide plenty of flexibility in how you can request the collection. The list of methods includes the following items:
- GetMessageQueueEnumerator. Provides a dynamic listing of public queues available on the network as a MessageQueueEnumerator object.
- GetPrivateQueuesByMachine. Retrieves all the private queues on the specified computer.
- GetPublicQueues. Basically the same as GetMessageQueueEnumerator, but instead returns a MessageQueue object.
- GetPublicQueuesByCategory. Filters the list of public queues on the network by category, a GUID value, which is a read/write property of a queue.
- GetPublicQueuesByLabel. Provides a static snapshot of public queues, filtered by a text label associated with each queue.
- GetPublicQueuesByMachine. Retrieves all the public queues that reside on the specified computer, as a MessageQueue object.
As you can see from the descriptions of these methods, a MessageQueue object is commonly used to retrieve another MessageQueue object that contains a different set of queues than the original object.
Retrieving a Queue's Messages
The MessageQueue class supports the MSMQ ability to either peek or receive messages. Peeking means that you examine one or more of the messages in a particular queue without removing it (or them) from the queue. This can be a handy way to take a look at the queue and implement logic for handling messages in a different order from the order they appear in the queue. (You can view the messages in order by message priority, and also in the order in which they arrived at the queue. A real-life parallel might be the different lines at the hotel check-in counter: one for the concierge floor, another for frequent visitors, and another for everyone else.)
Once the application peeks at a message on the queue, it can elect to receive the message, or it can directly receive it without peeking. Receiving a message means that it is popped off the queue and the application can do whatever is appropriate with the message. Once an application receives a message, that message is gone from the queue for good, unless some process returns it to the queue.
An application can peek or receive messages either synchronously or asynchronously using the MessageQueue object. It provides PeekCompleted and ReceiveCompleted events you can use to receive events asynchronously. Typically, an application using these events will specify a timeout to limit the duration an application sits waiting for messages.
For synchronous applications, the MessageEnumerator class provides references to a collection of messages on a queue.
As you work with MSMQ, it is helpful to monitor the messages in various queues used by an application. Microsoft provides two tools for this purpose: a Message Queues node in the Microsoft® Visual Studio® .NET IDE, and a Microsoft® Management Console (MMC) snap-in. Figure 2 shows the MMC snap-in.
Figure 2. MSMQ MMC Snap-in, shown here as part of the Computer Management utility in Windows 2000
There are various ways to start up this snap-in, but the easiest is to open the Windows 2000 Computer Management utility, expand the Services and Applications section of the tree view on the left, and select Message Queuing. This is also a great way to verify that MSMQ is installed on a particular machine.
As the name implies, the Message class lets you access and manipulate individual messages in a queue, as well as format and fine-tune a message that you add to a queue. In most cases, you'll use a Message object to receive a reference to a specific message as you loop through a collection of messages.
Message data is stored in the Body property of the Message object. The contents of the Body property are serialized when the message is sent, using the Formatter property you specify. The serialized contents are found in the BodyStream property. You can also set the BodyStream property directly, for example, to send a file as the data content of a message. You can change the Body or Formatter properties at any time before sending the message, and the data will be serialized appropriately when you call the Send method of the MessageQueue object.
The MessageEnumerator class is unique in that it proves very flexible access to the messages in a queue, as a dynamic collection of messages. It is the best means to process multiple messages, giving you the flexibility to peek or receive messages as necessary.
Methods of the MessageQueue class can return either a MessageEnumerator pointing to a dynamic list of messages in the queue or an array that contains a copy at a given instant, a kind of snapshot of the queue at the time the method was called.
An enumerator does not remove the messages from the queue when it queries the queue. It returns information about the message at the current cursor position, but it leaves the message in the queue.
A MessageEnumerator object is a cursor with references to messages in the order of the messages as they appear in the queue, according to message priority. Move the cursor to the first message in the queue by calling MoveNext. After the enumerator has been initialized, you can use MoveNext to step forward through the remaining messages, and can specify whether to wait for a message to become available by passing a timeout to the MoveNext method.
Because the enumerator is dynamic, a message that is appended beyond the cursor's current position (for example, due to low priority), can be accessed by the enumerator. A message that is inserted before the cursor's current position cannot be accessed. It is not possible to step backward with a MessageEnumerator because it allows forward movement only. But you can use the Reset method to move the cursor back at the beginning of the queue.
There are three recommended ways to reference a queue using the MessageQueue class and other classes in the Messaging namespace: by path, format name, and label. Using a path is the only option available when creating a new queue, because the other two reference options rely on properties of the queue that can be set only once the queue exists. When you reference an existing queue, any of the referencing options will work.
You can use a path to a queue in order to uniquely identify the computer and queue name for the queue. The path must be in the form servername\queuename, which, because queue names on a particular server must be unique, specifies a unique queue.
Table 1 shows the syntax for various forms of queue paths, specifying the Payroll queue located on the Salmon server. The names with a $ suffix identify system queues created when you install MSMQ on a server or client machine. Using a path to reference a queue requires providing both the server and queue name, as well as any qualifier to reference a system queue.
Table 1. Queue path syntax
|Private queue on local machine||.\Private$\Payroll|
|Machine journal queue||Salmon\Journal$|
|Machine dead-letter queue||Salmon\Deadletter$|
|Machine transactional dead-letter queue||Salmon\XactDeadletter$|
The Path property of the MessageQueue object is one way to reference a queue. Using a path reference, Visual Basic code can reference a queue like this:
msgQueue.Path = " Salmon\Private$\Payroll"
Format Name Reference
A format name is a unique identifier for a queue that is generated by MSMQ when the queue is created. The format name can also be generated later by the application. A format name reference is in the form of a string indicating whether a queue is public or private, followed by the GUID for the queue, and other identifiers as necessary. This is the most efficient way to reference a queue, since MSMQ doesn't have to interpret paths or labels to find the queue.
Table 2. Path information used for each type of queue
|Private queue||FORMATNAME:PRIVATE= MachineGUID\QueueNumber|
|Journal queue||FORMATNAME:PUBLIC= QueueGUID;JOURNAL or FORMATNAME:PRIVATE= MachineGUID\QueueNumber;JOURNAL|
Note that using the FORMATNAME keyword, you must specify whether the queue is public or private, along with the GUID—represented by "QueueGUID" in Table 2 —assigned by MSMQ when the queue was created.
Here is an example of using a format name with the Path property in Visual Basic:
msgQueue.Path = _ "FORMATNAME:PUBLIC=3d3dc813-c555-4fd3-8ce0-79d5b45e0d75"
In MSMQ 3.0, the queue's FormatName property lets you specify a public format name, private format name, distribution list format name, multicast address format name, multiple-element format name, or a queue alias.
A multi-element format name is a concatenation of any number of comma-delimited public format names, private format names, direct format names, multicast address format names, or distribution list format names. A MULTICAST formatname allows you to efficiently send a single copy of message such that it is received by multiple recipient queues while leveraging network support for reliable IP multicast. Direct format names, such as DIRECT=HTTP://URLAddressSpecification/msmq/QueueName let you reference public or private queues without accessing the directory.
A label is a descriptive and potentially non-unique name for the queue that is assigned by the queue administrator when the queue is created. Because labels are not guaranteed to be unique, you will receive an error if a name conflict exists when you try to connect to a specific queue using this reference method.
Labels can be useful in a situation where you know you are going to be moving a queue from one computer to another. If you refer to the queue by label only, all of your operations will continue to work successfully after the queue has been moved to its new location, provided that there are no other queues with that label on the new computer. If there are, sending a message to a queue will produce an error.
Here is an example of using a label reference with the Path property in Visual Basic:
The sample application that accompanies this white paper demonstrates several of the most common programming tasks using MSMQ through System.Messaging. Figure 3 shows the user interface to the application. Use it from the top down to list the queues on the local machine (it is simple to modify the code to allow for selecting other machines), select a queue to process and list properties about the queue, create a new queue, then examine the contents of the queue.
Figure 3. Sample System.Messaging application
How do I…
The remaining sections answer several "How do I…" questions about programming with System.Messaging.
Make it easier to use the System.Messaging classes?
Like any .NET application, you can import a namespace into your Visual Basic application and save repeatedly typing full namespace and class names.
In order to use this .NET namespace and have the benefit of Microsoft® IntelliSense® in the Visual Basic development environment, you have to add a reference to the namespace. To do this, select Project and then Add Reference… from the Visual Studio .NET main menu, select the .NET tab, and double-click on the System.Messaging.dll entry in the list. The Add Reference dialog should look like Figure 4. Click OK to commit the change to your project. System.Messaging will now appear in the Reference section of the Solution Explorer window.
Figure 4. The Visual Basic .NET Add Reference dialog after selecting the System.Messaging namespace
To import System.Messaging, include this line of code at the top of your code module:
Get a list of queues on a machine?
The sample application includes a List Available Queues button, which populates the adjacent combo box (cmbQueueList ) with all of the private queues on the local machine. Here is the code in the cmdListQueues Click event handler:
Private Sub cmdListQueues_Click( _ ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles cmdListQueues.Click Dim sMachineName As String = MachineName Dim PrivateQueues() As MessageQueue = _ MessageQueue.GetPrivateQueuesByMachine(sMachineName) Dim PrivateQueue As MessageQueue lblMachineName.Text = "Machine name: " & sMachineName cmbQueueList.Items.Clear() For Each PrivateQueue In PrivateQueues cmbQueueList.Items.Add(PrivateQueue.QueueName) Next End Sub
The name of the local machine (MachineName) is provided by the System.Environment namespace, imported at the beginning of the module. The machine name is used as part of the path reference for the queue. The code uses the GetPrivateQueuesByMachine method of the MessageQueue object to retrieve a MessageQueue object with the private queues. The rest of the code simply loops through the list of queues and adds the value of the QueueName property, for each queue, to the combo box.
Read and use properties of the queue?
As the main interface into the queues on the target server, the MessageQueue object makes available many properties that let an application gather information about the queue's capabilities. Some properties are read-only-set or specified when the queue is created-and others are read/write.
In the sample application, when the user selects a queue from the combo box, its SelectedIndexChanged event handler lists various queue properties in a label control on the form:
Private Sub cmbQueueList_SelectedIndexChanged( _ ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles cmbQueueList.SelectedIndexChanged mQueue = New MessageQueue(cmbQueueList.Text) lblQueueProperties.Text = _ "Can Read: " _ & mQueue.CanRead.TrueString & vbCrLf _ & "Can Write: " _ & mQueue.CanWrite.TrueString & vbCrLf _ & "Machine Name: " _ & mQueue.MachineName & vbCrLf _ & "Path: " _ & mQueue.Path & vbCrLf _ & "Queue Name: " _ & mQueue.QueueName & vbCrLf _ & "Transactional: " _ & mQueue.Transactional.TrueString & vbCrLf _ & "Use Journal Queue: " _ & mQueue.UseJournalQueue.TrueString & vbCrLf End Sub
In this case, the code simply uses the mQueue object variable, set to a reference to the selected queue, and reads various properties.
Note Some properties are only available when referencing a queue on a server rather than a queue on a local machine. If you try to read such a property from what the system calls a "workgroup installation computer," it will throw an exception that you can trap.
Peek at messages without removing them from the queue?
You can peek at messages in a queue synchronously, causing the application to stop and wait for a response from MSMQ before continuing with the next line of code. The sample application uses this technique, and it is suitable for many applications.
In this case, the code uses the GetAllMessages method of the MessageQueue object to return another MessageQueue object with the messages. Then, using an instance of the Message class, the code loops through any messages it finds and adds them to a list box.
Private Sub cmdPeekMessages_Click( _ ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles cmdPeekMessages.Click lstPeekMessages.Items.Clear() Try Dim msgMessages() As Message = _ mQueue.GetAllMessages() Dim msgMessage As Message lblPeekMessages.Text = "" For Each msgMessage In msgMessages lstPeekMessages.Items.Add(msgMessage.Body) Next Catch exc As Exception lstPeekMessages.Items.Add( _ "No messages in queue.") lblPeekMessages.Text = _ "Exception thrown: " & exc.Message End Try End Sub
Note The above code will work only if you are not using the XmlMessageFormatter class, used to serialize and de-serialize objects to or from the body of a message, using the XML format based on the XSD schema definition. In that case, you'd need to set up the TargetTypes property of XmlMessageFormatter.
It is possible that a queue might contain no messages when this code executes. In that case, the GetAllMessages method throws an exception that the code can trap. To avoid this, the procedure wraps the method call and subsequent code to loop through the messages in a Try/Catch block, demonstrating exception handling in .NET Framework programming.
Send a message?
Sending a message to a queue is about as easy an operation as it can be, although there are lots of options for formatting the body of the message.
If you are sending a string (or other serializable objects) as the body, simply pass the message as the parameter of the Send method of the MessageQueue object:
Private Sub cmdSendMessage_Click( _ ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles cmdSendMessage.Click Dim sMessage As String = txtMessageToSend.Text Try mQueue.Send(sMessage) lblSendMessage.Text = "Message added to queue." Catch exc As Exception lblSendMessage.Text = "Message not sent: " _ & exc.Message End Try End Sub
The System.Messaging namespace includes three formatting objects to support converting messages to and from various formats.
- ActiveXMessageFormatter. Serializes and deserializes primitives, classes, enumerations, and other objects into and from messages using binary format, in order to interoperate with the MSMQ COM objects.
- BinaryMessageFormatter. Serializes and deserializes objects into and from messages using binary format.
- XmlMessageFormatter. Serializes and deserializes objects from the body of a Message, using the XML format.
Create a queue?
The Create method of the MessageQueue object supports creating a new public or private queue on any machine on the network with MSMQ installed.
Remember that when creating a queue, you have to use a path reference to the queue. The code for the Click event procedure of the cmdCreateQueue command button creates a new private queue on the local machine using this path reference:
The code traps any exceptions raised. Normally, you'll catch exceptions because a queue of the specified name already exists or MSMQ isn't installed on the target machine.
Private Sub cmdCreateQueue_Click( _ ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles cmdCreateQueue.Click If txtNewQueueName.Text = sNameOfQueue _ Or txtNewQueueName.Text = "" Then txtNewQueueName.Focus() Else Try 'Create a new private queue Dim msgQueue As MessageQueue msgQueue = MessageQueue.Create( _ MachineName & "\Private$\" _ & txtNewQueueName.Text) lblCreatedQueue.Text = "Created " _ & txtNewQueueName.Text & "." Catch exc As Exception lblCreatedQueue.Text = _ "Unable to create new queue:" & exc.Message End Try End If End Sub
Peek at all messages in a queue?
The GetMessageEnumerator method of the MessageQueue object returns a MessageEnumerator object that you can use to loop through and examine messages. Even though the prefix "Get" in the method name might imply that this method removes messages from the queue, it actually peeks at the messages. This is a handy method for examining all the messages on a queue and deciding how to handle each message.
Once the code obtains a reference to the MessageEnumerator object, it can loop through the collection of messages and use the Current property to return a reference to the current message. In this case, the code adds the value of the Label property of the message to a list box.
The MoveNext method moves to the next message, each in turn, but it also moves to the first message when starting a loop.
Private Sub cmdGetAllMessages_Click( _ ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles cmdGetAllMessages.Click lstGetMessages.Items.Clear() Try Dim enumMessage As MessageEnumerator _ = mQueue.GetMessageEnumerator() While enumMessage.MoveNext() Dim msgNext As Message = enumMessage.Current lstGetMessages.Items.Add(msgNext.Label) End While enumMessage.Close() Catch exc As Exception lstGetMessages.Items.Add("No waiting messages.") lblGetMessages.Text = "Exception thrown: " _ & exc.Message End Try End Sub
Process a collection of messages, removing them from the queue?
You can also use the MessageEnumerator object to process messages and actually remove them from the queue. Peeking is useful for implementing logic for message handling, but most often an application just needs to grab a message, process it, and get on with life.
The main difference between the code shown below and the previous example is that it uses the MessageEnumerator's RemoveCurrent method instead of the Current property.
Private Sub cmdProcessMessage_Click( _ ByVal sender As System.Object, _ ByVal e As System.EventArgs) _ Handles cmdProcessMessage.Click Try Static enumMessage As MessageEnumerator _ = mQueue.GetMessageEnumerator() 'Move to the first message, if any enumMessage.MoveNext() Dim msgNext As Message msgNext = enumMessage.RemoveCurrent() txtProcessMessage.Text = msgNext.Label _ & vbCrLf & msgNext.Body lblProcessMessage.Text = "" 'Reset the message ennumerator. If any messages 'arrive with higher priority than the last message, 'this will retrieve the new message first. enumMessage.Reset() Catch exc As Exception txtProcessMessage.Text = "No messages can be " _ & "retreived " & "from the " _ & cmbQueueList.Text & " queue." lblProcessMessage.Text = "Exception thrown: " _ & exc.Message End Try End Sub
Calling the Reset method of the MessageEnumerator object is necessary in some circumstances. This object provides a forward-only cursor—like in database programming—for the messages in a queue. If any messages are added to the queue while processing the current message, when the new message has a higher priority than the current message, the application won't process the new message. Doing so would require moving backward in the cursor.
The Reset method causes the current enumerator to point to the beginning of the message queue. Using Reset is not appropriate for all applications, and can hurt performance, but is useful if an application needs to process all messages.
You have now examined how to use the System.Messaging namespace to implement Microsoft Message Queue in your applications. Although many classes, interfaces, and delegations are at your disposal, a small handful will serve most application needs. The rich set of objects means that there are almost always several ways to accomplish the same task:
- The MessageQueue class is the main entrèe into MSMQ for an application. Through its interface, you have access to all the queues across the network, can read and write to any queue for which you have the proper permissions, and maintain queues.
- The Message class lets you work with individual messages when both reading and writing to a queue. MSMQ allows plenty of latitude for message format, so you can match it to the needs of your application.
- The MessageEnumerator is ideal for working with sets of messages in a queue. You can both peek and receive messages, and use various objects to manage messages.
Because the objects in the System.Messaging namespace encapsulate so many of the MSMQ features, code for most tasks is simple and straightforward.
Don Kiely has authored and co-authored several programming books, including Visual Basic Programmer's Guide to the Windows Registry from Mabry Software, and writes for several industry journals including InformationWeek, EarthWeb, and Visual Basic Programmer's Journal. He is a Visual Basic, SQL Server, XML and ASP instructor for Application Developers Training Company. To round out his day, he is Software Technologist for Third Sector Technologies in Fairbanks, Alaska.
About Informant Communications Group
Informant Communications Group, Inc. (www.informant.com) is a diversified media company focused on the information technology sector. Specializing in software development publications, conferences, catalog publishing and Web sites, ICG was founded in 1990. With offices in the United States and the United Kingdom, ICG has served as a respected media and marketing content integrator, satisfying the burgeoning appetite of IT professionals for quality technical information.
Copyright © 2001 Informant Communications Group and Microsoft Corporation
Technical Editing by PDSA, Inc. and KNG Consulting