Export (0) Print
Expand All

Point-to-Point Message Queues with the .NET Compact Framework

.NET Compact Framework 1.0
 

Daniel Moth
Trend Control Systems Ltd.

July 2005

Applies to:
   .NET Compact Framework
   Visual Studio .NET 2003
   Windows CE .NET version 4.2
   Windows CE version 5.0
   Windows Mobile 2003

Summary: This article explains how to write and use a managed wrapper for the message queue point-to-point functions that are unique to Windows CE .NET and later. (15 printed pages)


Download Point to Point Message Queues with the .NET Compact Framework.msi from the Microsoft Download Center.

Contents

Introduction
Overview of Native API
Designing the Managed Interface
P2PMessageQueue
Using P2PMessageQueue
Conclusion
About the Author

Introduction

Are you frustrated by the limitations of interprocess communication (IPC) on Windows CE devices? On other Windows platforms, IPC is achieved by means of named pipes (native) or remoting (managed); none of these options are available to Windows CE. This article describes point-to-point message queues — a little-known IPC mechanism that is efficient, flexible, and unique to Windows CE version 4.0 and later. Furthermore, this article also describes how you can design and use a managed wrapper, which makes IPC in .NET Compact Framework applications extremely easy.

Developers can use point-to-point message queues in many ways. Developers commonly use IPC as a workaround for the inability to host the .NET Compact Framework runtime (the native process communicates through IPC with the managed process), and point-to-point message queues are the perfect choices. Point-to-point message queues also interact with the operating system, for example, for getting power information. This article will not examine these other uses of point-to-point message queues, but you should be able to explore the other uses after you read about the core principles.

Overview of Native API

This section describes the point-to-point message queue functions in one place for easy reference. For an MSDN view of the six functions and two structures in this application programming interface (API), see Message Queue Point-to-Point Reference.

When a developer uses IPC, one process creates a queue for writing, and the other process creates a queue for reading (CreateMsgQueue). During bidirectional IPC communication, developers need two queues on each side of the communication (in two separate processes or between two objects in the same process).

Developers use several methods when working with point-to-point message queues. For example, there are methods for reading (ReadMsgQueue), writing (WriteMsgQueue), and closing the queue (CloseMsgQueue). Also, developers can open an existing queue (OpenMsgQueue) if, in addition to the queue handle, they have the handle to the source process that owns the queue. Developers can query some statistics (GetMsgQueueInfo) and, finally, developers can make their applications wait for the handle of a queue (WaitForSingleObject) to be signaled to determine if there is data in the queue.

When opening an existing queue, the only option developers can specify is whether the queue is for reading or writing. When creating a queue, developers can specify additional parameters: name (lpszName) and options (lpOptions of type MSGQUEUOPTIONS). Queue options include the following: maximum size of a single message (cbMaxMessage), maximum number of messages (dwMaxMessages) in the queue at any one time, whether buffers should be dynamically allocated (dwFlags, MSGQUEUE_NOPRECOMMIT), and whether readers can exist without writers and vice-versa (dwFlags, MSGQUEUE_ALLOW_BROKEN). When closing a queue, developers use the handle as an argument (hMsgQ).

Writing to a queue takes a pointer (lpBuffer), the number of bytes in the message (cbDataSize), a timeout (dwTimeout), and whether the message is an alert message (dwFlags, MSGQUEUE_MSGALERT). Reading from a queue requires that you pass in a buffer (lpBuffer), the buffer's size (cbBufferSize), and a timeout value (dwTimeout); in return, you get the actual number of bytes read (lpNumberOfBytesRead) and whether the message was an alert (pdwFlags, MSGQUEUE_MSGALERT). Note that the timeouts are in milliseconds; 0 is interpreted as "do not block," whereas INFINITE (-1) is interpreted as "block until the operation completes or the status of the queue changes." Both the read and write functions return TRUE if they succeed; otherwise, they return FALSE. In the latter case, you can get extended error information (GetLastError). The possible reasons for the functions to return FALSE are: the buffer was too small (ERROR_INSUFFICIENT_BUFFER); there are no writers or readers, and the developer did not specify MSGQUEUE_ALLOW_BROKEN as stated in the previous paragraph (ERROR_PIPE_NOT_CONNECTED); or a timeout occurred (ERROR_TIMEOUT). In the case of WriteMsgQueue, you may also get the error, ERROR_OUTOFMEMORY, if you did not specify MSGQUEUE_NOPRECOMMIT as alluded to in the previous paragraph.

You can call GetMsgQueueInfo to obtain a structure (MSGQUEUEINFO) that holds statistical information. The information that the structure contains includes some of the parameters passed in when the developer created the queue: maximum size of message (cbMaxMessage), maximum number of messages (dwMaxMessages), whether buffers should be dynamically allocated (dwFlags, MSGQUEUE_NOPRECOMMIT), and whether readers can exist without writers and vice-versa (dwFlags, MSGQUEUE_ALLOW_BROKEN). In addition the structure contains the following: the number of non-alert messages that are currently in the queue (dwCurrentMessages), the maximum number of messages that have ever been in the queue (dwMaxQueueMessages), the number of current readers (wNumReaders), and the number of current writers (wNumWriters).

Designing the Managed Interface

The first thing to do before you design a managed interface for any native API set is to see if it has been implemented already. In the case of point-to-point message queues, there is no existing library and search results offer a small amount of information about the API descriptions. Since there is no prior work in this area, it seems worth it to describe a design and implementation for a managed wrapper to the point-to-point message API.

There is a common pattern that emerges when developers wrap a set of related Win32 APIs into one or more managed classes; most Windows APIs operate on, and expect as their first argument, a handle. From an object-oriented perspective, you can think of the handle as the object identity. You can group all handle-related methods into a class that exposes the same methods. The methods have the same signature as the original methods, except they do not expect the handle. The handle is obtained on object creation and closed on object disposal/destruction. A handle is represented in .NET Compact Framework by the IntPtr type. So, following the previous information, you can create a class that wraps the specific native API like this the following.

+ constructor(String name, MsgQueueOptions opt)
+ ReadMsgQueue(byte[] buf, Int32 bufSize, Int32 numRead, Int32 timeout, Int32 flags)
+ WriteMsgQueue(byte[] buf, Int32 bufSize, Int32 timeout, Int32 flags)
+ GetMsgQueueInfo(MsgQueueInfo info)
+ Close()

The five methods in the previous code example could form the main interface, but you would, of course, have to define MsgQueueOptions and any other structure in managed code (like you would do for any unmanaged structure that is used in platform invoke). This interface is a good start and it gives you a wrapper, but it isn't very object-oriented, it doesn't fit with the rest of the .NET Compact Framework, and it still puts a larger burden than necessary on the client side (in terms of code to be written). The interface can still be improved.

Wherever you can introduce overloads of methods, you should do so to simplify the methods that the client code has to deal with. For example, if you need to create a queue without a name, the client should not have to pass in NULL — rather the name parameter should not be in the constructor. If you need to read or write to the queue in a blocking fashion, rather than pass INFINITE (-1) as the timeout argument, you should not have to pass anything. Applying these changes will augment the number of methods in your class and will make it easier to work with in the simpler scenarios, while not limiting more complex situations.

The interface can be refactored further. Note how you need to pass a byte array along with the size of the array; passing the array size should not be necessary because you can determine the size of a given array at any given time. Unless you explicitly require the array length in your signature (for example, for performance reasons because caching the size, rather than recalculating it every time, will be faster), you can remove the bufSize parameter. In fact, you can encapsulate the byte array parameter (buf) into its own type/class, in combination with the flags parameter (which denotes whether the message is an alert message or not).

Another technique for making the API more object oriented is to eliminate structures — where it makes sense. For example, rather than having to set up a structure and pass it to the constructor, you can inline the structure elements as parameters with suitable overloads (in other words, you flatten the structure members as a series of parameters). Rather than return a structure with the queue information, a better solution is to inline the structure fields into read-only properties on the class itself.

You should recall these two facts from the first section of this article: point-to-point message queue API methods, like so many others, return a BOOL to indicate success or failure, and extended error information requires a separate call. You can take two non-exclusive approaches for mapping this behavior (that is, the Boolean return and the need to separately retrieve the extended error information). The first approach is to have the application throw an exception when a failure occurs and have the exception carry the extended error information. The second approach is to have the application return an enumeration that contains the possible outcomes of the method call (including success). As a general guideline, develop applications so that they throw exceptions only when the situation is unrecoverable.

At this stage, it is worth looking for similar classes in the .NET Framework, and indeed you will find the MSMQ classes in System.Messaging (also available in the .NET Compact Framework version 2.0). You can adopt the same naming convention used by the members of that class (for example, change Read to Receive, and Write to Send). Note how the MSMQ class offers a Purge method for emptying the messages in the queue. You can enhance your class with this method; in other words, just because the native API doesn't offer a method, it doesn't mean you cannot add value to your wrapper by adding one.

As the OpenMsgQueue method returns a handle and you have mapped the handle to a class, it makes sense that your wrapper method returns an instance of your wrapper class. Also, the method requires no existing state to execute, so you should make it static. Finally, you need to flatten the structure into the single argument that is required: whether it is a read-only or write-only queue.

Note that the platform invoke declarations are not described here; you can find them in the attached code. The next section shows the results of all of the previously described refactorings (plus many more). It also describes the final addition to the class: the DataOnQueueChanged event!

P2PMessageQueue

The best way to examine the public interface of P2PMessageQueue is by using the class. IntelliSense will show the descriptive help for each method and each parameter. Because the full code is included in the download for this article, you can view the source. Now is a perfect time to do download and familiarize yourself with the source. The following paragraphs will sum up the P2PMessageQueue interface.

Figure 1 shows the Unified Modeling Language (UML) representation of the class (and its two dependencies: the Message class and the ReadWriteResult enumeration).

Click here for larger image

Figure 1. A class diagram showing the three types: P2PMessageQueue, Message, and ReadWriteResult. Click the thumbnail for a larger image.

If you prefer a less descriptive screenshot (but more familiar for some people) of the Object Browser, see Figure 2.

Click here for larger image

Figure 2. Visual Studio Object Browser showing the three types. Click the thumbnail for a larger image.

Before you examine the samples in the next sections of this article, you should take note of items that haven't been discussed and that are not visible in the UML diagram (on purpose).

A DataOnQueueChanged event has been added. As you'll see, the P2PMessageQueue class is usable without it, but this enhancement will notify the client when a read-only queue is not empty and when a write-only queue is not full. Internally, you spin a thread that waits on the queue handle and raises the event when the handle is signaled, which takes care of false positives. (See the code for details.) Note that the event will not be on the UI thread, so you need to use Control.Invoke. Looking at the code, you will also see how the thread cleanly shuts down when the Close method is called, which is something required with .NET Compact Framework version 1.0 threads. The Thread class in the .NET Compact Framework version 1.0 does not provide the IsBackground property or Abort method; therefore, you must be sure that your application provides the code necessary to cleanly terminate all threads that the application started. If you don't provide this code, the process running your application will likely be unable to terminate because a running .NET Compact Framework 1.0 thread keeps the process running even though the main application thread attempts to exit.

Also, if you look at the implementation, you will find two protected virtual methods.

# void StartEventThread()
# Int32 GetBehaviourFlag()

You may override the first method with an empty method, if you do not require the event thread to start. Not wanting the event thread to start is an uncommon scenario (because you can simply not subscribe to the event), but the option is there. The second method returns the integer argument used in the flag passed to the CreateMsgQueue method (MSGQUEUOPTIONS.dwFlags). The default of MSGQUEUE_ALLOW_BROKEN is fine, but if you want to change it (for example, to 0 or MSGQUEUE_NOPRECOMMIT), overriding GetBehaviorFlag is the way to do it.

Now you should have a good understanding of what the P2PMessageQueue class looks like and why it was designed the way it is. Next, you'll see how to use the components.

Using P2PMessageQueue

In this section, you will look at examples for using the P2PMessageQueue class and related types.

Note   When you run the sample, you have the option of deploying to a Windows CE or Pocket PC 2003 emulator or device. You can debug the project (and you can run the application) on either platform with no modifications. The first screen that you see when you run the sample prompts you to choose between a reader and a sender process. Whichever you choose, you will have to run the .exe file (from Program Files) a second time, and then select the other option. On the Pocket PC platform, whether using the emulator or device, you must rename the .exe file (otherwise the existing running application gets activated).

Simple IPC Exchange of Strings

You begin with a managed process that passes strings to another .NET Compact Framework application (you could also make the sender or receiver a native application). There are three different ways that you can read the string on the receiving end (same principles apply for the sending side): blocking, non-blocking, and event driven.

The sender and receiver's graphical user interfaces (GUIs) should be self-describing in terms of functionality, as shown in Figures 3, 4, and 5.

Figure 3. Main form

Figure 4. Reading side

When the sender taps the Send button, the sender sends a string (as entered in the textbox) and optionally sets the message as an alert message (based on the Message Is Alert check box state). The sender blocks until the message is sent for the specified timeout (as the sender selected in the combobox). The label at the bottom of the form under the check box shows the returned result of the Send method, as shown in Figure 5 (that is, OK).

Figure 5. Sending/writing side

The Send method from the download sample is repeated here.

private void cmdSend_Click(object sender, System.EventArgs e) {
Message msg;

msg = new Message(
System.Text.Encoding.ASCII.GetBytes(txtSend.Text), 
chkIsAlert.Checked);

    ReadWriteResult rwr = mQue.Send(msg, mTimeout);
    lblSendResult.Text = rwr.ToString();
}

After the reader receives a message, it is displayed in the listview (third column) and whether it is an alert message or not (second column). The first column will always read OK when the message is successfully received. By default, to receive a message, tap the Receive button; if there is no message to receive or if the method fails, the listvew's first column will indicate the reason (the other two columns are not applicable in this scenario).

In both of the reading and sending cases, the Queue Info menu (tap Info, and then tap Queue Info) displays data about the queue. The Mode menu (tap Read, and then tap Mode) on the receiving end has three menu items: On Demand Only, Event driven, and Block a Thread. These items configure how the program receives messages off the queue. After you chose a mode, it should not change for the lifetime of the sample application (developers can mix and match as they see fit for their own designs). The following subsections describe the three read modes.

Read on demand (corresponding to the menu On Demand Only)

When the receiver taps the Receive button, the following method executes.

private void cmdReceive_Click(object sender, System.EventArgs e) {
    Message msg;
    msg = new Message();

    // mTimeout is set by the end user by means of the GUI 
    // to DON'T BLOCK (0), BLOCK (-1), or a real timeout value
    ReadWriteResult rwr = mQue.Receive(ref msg, mTimeout);

    ListViewItem lvi;
if (rwr == ReadWriteResult.OK){ 
    bool isAlrt;
        string payload;
        isAlrt = msg.IsAlert;

        byte[] bytes = msg.MessageBytes;
        payload = System.Text.Encoding.ASCII.GetString(
                bytes, 0, bytes.GetLength(0));

        lvi = new ListViewItem(
                new string[]{rwr.ToString(), isAlrt.ToString(), payload});
}else{      
        lvi = new ListViewItem(
                new string[]{rwr.ToString(), @"n\a", @"n\a"});
    }
        listView1.Items.Add(lvi);
        listView1.Columns[2].Width = -2;}

Event Driven

The event driven model basically means that the application does not poll for new messages by calling Receive at arbitrary moments (for example, on a timer or by requiring the user to tap the Receive button), but instead the application subscribes to and catches the event from the P2PMessageQueue class. To subscribe to the event, you use the normal .NET Compact Framework delegate idiom (creation of the queue is the same regardless).

mQue = new P2PMessageQueue(
    isReader, txtQueueName.Text, maxL, maxM, out firstTime);
mQue.DataOnQueueChanged += new EventHandler(mQue_DataOnQueueChanged);

When the event is raised, the method gets called and, in this case, simply invokes the existing receive method.

private void mQue_DataOnQueueChanged(object sender, EventArgs e) {
    this.Invoke(new EventHandler(this.cmdReceive_Click));
}

Block a Thread

A third way of reading from the queue is by creating a thread and letting it block waiting for the queue's Receive method. Every time a message is received, the application processes it and then loops back to blocking again. Here is some sample code with some explanations.

Somewhere, you create and start the thread.

    Thread t = new Thread(new ThreadStart(ThreadBlockedOnRead));
    t.Start();

The thread runs in the following method. (For more context, you can download the code).

private void ThreadBlockedOnRead(){
    while (mMode == 2){ // Thread mode
        Message msg = new Message();

        //Can actually omit a timeout for a true infinite block 
        ReadWriteResult rwr = mQue.Receive(msg, 60 * 1000);
        if (rwr == ReadWriteResult.InvalidHandle || mMode != 2){
            return;
        }
                
        string body = rwr.ToString();
        if (rwr == ReadWriteResult.OK){
            byte[] bytes = msg.MessageBytes;
            string bytesAsString = 
System.Text.Encoding.ASCII.GetString(
bytes, 0, bytes.GetLength(0));
            body += " | " + msg.IsAlert.ToString() + " | " + bytesAsString;
        }

        MessageBox.Show(body, "To terminate this mode use the menu again");
    }
}    

This concludes the basic sample code. The three ways of reading the queues can be applied to sending through the queue as well. You could block, send when an event is signaled (that is, the queue transitions from full to not full), or attempt to send at arbitrary moments without blocking. In the next section, you will see how the design allows for messages on the queue to contain other structures — not just strings.

Note   When developing an application that relies on the event signaling to indicate that the queue has transitioned from full to not full, you need to perform an initial write to the queue when the application starts. If this initial write is not performed, the application will never begin writing because the initial write must be specifically performed to fill the queue for the queue to transition to the not full state and, therefore, signal the event that triggers further writes to the queue.

Sending and Receiving More Complex Types (Not Just Strings)

In the previous sections, you passed strings between the applications, but you can also pass any data type presuming you can convert it to and from a byte array. So you would convert the type to a byte array, create the Message class from it, and then send the Message. On the receiving side, you retrieve the Message, get the byte array, and then create the type from it (for example, expose the ToBytes and FromBytes methods on your type). Another approach is to inherit your own class from Message class and implement the conversion in it. Naturally, if you are trying to pass a complex object graph, the process of converting the types to and from byte arrays becomes considerably harder. Attempting to work with types for which you do not have the source code is of special concern because you may not have access to the type's complete state (for example, private members), and, therefore, you may not be able to accurately convert the type to and from a byte array.

To keep it simple, assume that you want to pass an Int64 and a Boolean from one process to the other. You would create a CustomMessage class that looks like the following.

    public class CustomMessage : Message {
        public long TotalMemory;
        public bool AfterGC;
        public CustomMessage(){
            TotalMemory = 0;
            AfterGC = false;
        }
        public CustomMessage(long totMem, bool afterGarbCol){
            TotalMemory = totMem;
            AfterGC = afterGarbCol;
        }
        public override byte[] MessageBytes {
            get {
                byte[] b1 = BitConverter.GetBytes(TotalMemory);
                byte[] b2 = BitConverter.GetBytes(AfterGC);

                byte[] b = new byte[9];
                Buffer.BlockCopy(b1, 0, b, 0, 8);
                Buffer.BlockCopy(b2, 0, b, 8, 1);
                return b;
            }
            set {
                TotalMemory = BitConverter.ToInt64(value, 0);
                AfterGC = BitConverter.ToBoolean(value, 8);
                base.MessageBytes = value;
            }
        }
    }

You add the two fields of interest (TotalMemory, AfterGC), override the MessageBytes property to implement your conversion (the get and set is where the conversion takes place), and then add a default constructor (the parameterized constructor is optional).

Now using the previous sample, you only have to change two places:

  1. When sending the message, you now create a CustomMessage instead by assigning values to TotalMemory and AfterGC.
    //msg = new Message(. . .); //Instead of this line
    msg = new CustomMessage(GC.GetTotalMemory(false), false);
    ReadWriteResult rwr = mQue.Send(msg, mTimeout);
    
    
  2. When receiving a message, you create a CustomMessage.
    //msg = new Message();
    msg = new CustomMessage(); //msg still declared as Message
    ReadWriteResult rwr = mQue.Receive(msg, mTimeout);
    
    

Then, you read its properties when mQue.Receive returns.

//byte[] bytes = msg.MessageBytes;
//payload = System.Text.Encoding.ASCII.GetString(. . .);
payload = "Total Memory = " + ((CustomMessage)msg).TotalMemory.ToString() + 
                    (((CustomMessage)msg).AfterGC ? " after a GC" : " without forcing a GC");

Exercises for the Reader

By now the reader should have a good understanding about both the class and its usage for IPC.

Queues are required for communicating with the operating system under some circumstances (the Windows CE operating system itself uses the mechanism internally in many places). For example, point-to-point message queues are used in the context of device power management (RequestPowerNotifications). By using the P2PMessageQueue class and your skills for converting the power management structures (for example, POWER_BROADCAST) to and from byte arrays, you can now receive and interpret such events. A full example is outside the scope of this article, but it may be the subject of another article.

Also outside the scope of this article is an example of an application written in eMbedded Visual C++ that communicates with a managed application. The principles are identical, except you use the P2PMessageQueue class on the managed side only.

Conclusion

The main idea to take away from this article is the actual use of the P2PMessageQueue class itself (and Message and ReadWriteResult). You can use it as is in your .NET Compact Framework applications for communicating with other processes — whether those applications are managed or native. Finally, if you are wrapping a native API into a managed class, the process you followed with the P2PMessageQueue class can serve as an example for the design decisions you face.

About the Author

Daniel Moth is a .NET Compact Framework MVP currently employed by Trend Control Systems Ltd. He can be reached through his Web log and appreciates feedback about his articles and blog entries. He replies to general .NET Compact Framework questions on the microsoft.public.dotnet.framework.compactframework newsgroup and the new MSDN forums.

Show:
© 2014 Microsoft