February 2010

Volume 25 Number 02

Robotics - Writing and Testing VPL Services for Serial Communication

By Trevor Taylor | February 2010

Microsoft Robotics Developer Studio (RDS) is, as you’d expect, a platform for programming robots. RDS first shipped in 2006 and the latest version, RDS 2008 R2, was released in June 2009.

RDS consists of four major components: the Concurrency and Coordination Runtime (CCR), Decentralized Software Services (DSS), Visual Programming Language (VPL) and Visual Simulation Environment (VSE). Sara Morgan wrote about the VSE in the June 2008 issue of MSDN Magazine (msdn.microsoft.com/magazine/cc546547).

VPL, however, is more pertinent to this article. VPL is a dataflow language, which means you create programs by drawing diagrams on the screen. At run time, messages flow from one block to another in the diagram, and this dataflow is effectively the execution path of the program. Because VPL is built on top of CCR and DSS, dataflows can occur asynchronously (as a result of notifications from services) and can also run in parallel. While VPL is intended for novice programmers, experienced coders can also find it useful for for prototyping.

This article outlines a simple RDS service that allows you to send and receive data using a serial port (also known as a COM port). The example code illustrates some of the key concepts involved in writing reusable RDS services.

To learn more about RDS and download the platform, go to microsoft.com/robotics. The package includes a useful help file, which is also available at msdn.microsoft.com/library/dd936006. You can get further help by posting questions to the various Robotics Discussion Forums at social.msdn.microsoft.com/Forums/en-us/category/robotics.

RDS Services

RDS services are built using CCR and DSS. They are conceptually similar to Web services because RDS has a service-oriented architecture (SOA) based on a Representational State Transfer (REST) model using the Decentralized Software Services Protocol (DSSP) for communication between services. Wading through all this alphabet soup, what this means is that you don’t talk directly to RDS services. Instead,  you send messages to a proxy, which acts as the external interface to the service (an approach Web developers will be familiar with). It also means that services can be distributed anywhere on the network.

Using a proxy has two effects. First, messages sent between services are serialized before transmission and deserialized at the other end. XML is the usual format for the data being transmitted. Second, the proxy defines a contract—effectively the set of APIs that are exposed to other services.

Every service has an internal state, which you can retrieve or modify by operations on the service. These involve sending a request message and waiting for a response message in a process similar to that used by most Web services.

When a service’s state changes, it can send out notifications to subscribers. This publish-and-subscribe approach makes RDS services different from traditional Web services because notification messages are sent asynchronously to subscribers.

When you build a new service, it automatically becomes visible in VPL and you can start using it immediately. This is one of the key RDS features, and it makes testing and prototyping very easy—you don’t have to write test harnesses in C#, because you can use VPL instead.

Controlling a Robot Remotely

Many simple educational robots have 8- or 16-bit microcontrollers for their on-board brains. But because RDS runs under the .NET Framework on Windows, it doesn’t generate code that can run directly on these robots. They must be controlled remotely via a communications link, instead. (The alternative is to have an on-board PC, such as the MobileRobots Pioneer 3DX).

Since the majority of microcontrollers support serial ports, a serial connection is the obvious solution. However, supplying the link using a cable isn’t ideal—it limits the robot’s mobility.

As an alternative, you can use Bluetooth to create a wireless connection by installing a serial-to-Bluetooth device on the robot. Some robots, such as the LEGO NXT, come with Bluetooth built in. Others, such as the RoboticsConnection Stinger, have optional Bluetooth modules. Bluetooth is a good choice, given that it’s standard on most laptops these days. Even if your PC doesn’t have Bluetooth, you’ll find inexpensive, readily available USB Bluetooth devices.

The good news is that you don’t have to know anything about programming Bluetooth because the connection to the robot will appear as a virtual serial port. You can use the same code as you would if a physical cable were providing the serial connection. The only difference is that you have to establish a pairing between your PC and the Bluetooth device on the robot so a virtual COM port will be created.

Some robot controller boards have firmware that can accept commands using a simple command language. For example, with the Serializer from RoboticsConnection (which gets its name from its use of a serial protocol), you can type commands to the controller via a terminal program like HyperTerminal. Commands are all human-readable, and you terminate each by pressing Enter.

If you’re designing your own protocol for use with a robot, you need to make a few choices. First, you must decide whether you’ll send binary data. Converting binary values to hexadecimal or decimal for transmission requires more bandwidth and increases the processing overhead for the on-board CPU. On the other hand, it makes reading the messages much easier, and you won’t experience any strange behavior due to misinterpreted control characters.

The next question is whether you want to use fixed-length packets or a more flexible variable-length format. Fixed-length is easier to parse and works best with hexadecimal.

You should also consider whether you’ll use a checksum. For computer-to-computer communication calculating check digits is easy. If, however, you want to test your robot by typing in commands manually, figuring out the check digits gets very tricky. When using checksums, 
typically the receiver sends back an acknowledgement (ACK) or negative acknowledgement (NAK) depending on whether the command came across successfully or not. Your decision about using a checksum really comes down to the reliability of the connection.

The SerialPort Service

It should be apparent by now why a serial-port service would be useful. The RDS package, however, doesn’t include such a service, although many of the robot samples use serial communication links. If you explore the RDS sample code, you’ll find that each example handles the serial port differently. This article outlines one approach to using a serial port. It’s not the only way to do it and not necessarily the best way.

Before going any further, make sure you’ve downloaded and installed RDS. The download for this article contains the source code of the 
SerialPort service. Unzip it into a folder under your RDS installation directory (also known as the mount point). Note that you should keep your code separate from the samples folder that comes with RDS so that you don’t mix up your code with the Microsoft code. Also, I recommend placing your code under the RDS mount point rather than elsewhere on your hard drive, because it simplifies development. I have a Projects folder where I keep all my own code in subfolders, making it easy to back up.

The main files in the SerialPort source code are SerialPort.cs, SerialPortTypes.cs and SerialPort.manifest.xml.

SerialPort.cs holds the implementation of the service, or the behavior. It consists of the service class itself, which contains the operation handlers, plus any required supporting methods and variables.

SerialPortTypes.cs contains the definitions of the service state, the operation types, and message types. In effect, it describes the interface to the service, and it should not contain any executable code. This is a key point about a service contract—it’s a data definition only.

SerialPort.manifest.xml is called the manifest and describes how services are combined, or orchestrated, to run an application. This file is used as input to the DssHost utility, which creates a DSS node that services run inside. In this case the manifest only runs the SerialPort service, which is fairly useless. A useful manifest would also specify other services that rely on the SerialPort service to make connections.

Before using the SerialPort service, you must run the DssProjectMigration tool and then recompile the service. Open a DSS Command Prompt (look under RDS in the Start menu) and make sure your paths are set up so you can execute files in the \bin folder. Then change to the directory where you unzipped the code and enter the following command:

Dssprojectmigration /b- .

The /b- option means don’t create a backup copy and the period character (.) refers to the current directory.

You can compile the service in Visual Studio or do so from the command line by typing:

Msbuild serialport.sln

Compiling the service generates the service DLL, a proxy DLL, and a transform DLL (which translates data types between the service DLL and the proxy). These will all be placed into the \bin folder under the RDS mountpoint. You should also keep the manifest and config files together by copying them into the \samples\config folder (though this isn’t essential).

The Service Contract

Every service has a unique contract identifier, which is declared in the Types.cs file and looks like a URL. Note, however, that it has no significance on the Web, and using a URL-style identifier is just a convenient way to guarantee uniqueness by allowing organizations to create their own namespaces. (For your projects you’ll need to use a namespace other than microsoft.com/robotics.) SerialPortTypes.cs contains the following definition:

public sealed class Contract {
  [DataMember]
  public const string Identifier = "https://www.promrds.com/contracts/2009/12/serialport.html";
}

A service contract also includes the state and operations that define what properties other services can manipulate and how. SerialPortState (see Figure 1) includes the serial port configuration, some parameters for timeouts, and the last byte received when operating asynchronously.

Figure 1 SerialPortState

[DataContract]
public class SerialPortState {
  private bool _openOnStart;
  private bool _asynchronous;
  private SerialPortConfig _config;
  private byte _lastByteReceived;
  private bool _isOpen;

  // Open the COM port when the service starts
  // Must be set in config file
  [DataMember]
  public bool OpenOnStart {
    get { return _openOnStart; }
    set { _openOnStart = value; }
  }

  // Operate in Asynchronous mode 
  [DataMember]
  public bool Asynchronous {
    get { return _asynchronous; }
    set { _asynchronous = value; }
  }

  // Configuration parameters for the serial port
  [DataMember]
  public SerialPortConfig Config {
    get { return _config; }
    set { _config = value; }
  }

  // Last byte received from the serial port
  [DataMember]
  public byte LastByteReceived {
    get { return _lastByteReceived; }
     set { _lastByteReceived = value; }
  }

  // Indicates if the port is currently open
  [DataMember]
  public bool IsOpen {
    get { return _isOpen; }
    set { _isOpen = value; }
  }
}

You can define your own data types for use in the state and message types. In this service, there’s a SerialPortConfig class. To make it visible in the proxy, it must be public and marked with the [DataContract] attribute. Each property in the class must be declared public and also be tagged using the [DataMember] attribute. If this isn’t done, the property will not be accessible.

You can also expose public enums—that way other programmers don’t have to use magic numbers in code that uses your service. It’s good programming practice, too, because it allows datatype checking.

When you design a service, you also have to decide how other services will interact with it. This SerialPort service supports the following operations, listed below in logical groups:

• Get, Subscribe
• ReceiveByte
• SetConfig, Open, Close, ClearBuffers
• ReadByte, ReadByteArray, ReadString
• WriteByte, WriteByteArray, WriteString

Note that the ReceiveByte operation is for internal use by the service itself and shouldn’t be used by other services. More on this later.

The Read and Write operations are all synchronous—the service does not respond until the operation has completed. If you open the serial port in asynchronous mode (explained below), the service will send you a ReceiveByte notification for every byte received and you should not use the Read operations. The Write operations, however, are always synchronous.

Each operation has a request message type and a response message type. Some of these might have properties and others will be empty—the datatype itself is sufficient to convey the appropriate message.

Service Behavior

As noted earlier, the executable code of the service is in SerialPort.cs. All services are derived from DsspServiceBase, which provides a number of helper methods and properties. When a service is started, its Start method is called:

protected override void Start() {
  InitializeState();
  base.Start();
  if (_state.OpenOnStart)
    OpenPort();
}

Start takes care of initializing the state (if necessary) and then opening the serial port automatically if OpenOnStart was specified in the config file (explained below).

Each operation supported by a service must have a service handler. But some operations, such as Drop and Get, are handled by the infrastructure (unless you wish to override the default behavior).

The OpenHandler shows a very simple handler:

[ServiceHandler]
public void OpenHandler(Open open) {
  // Remember the Asynchronous flag
  _state.Asynchronous = open.Body.Asynchronous;
  if (OpenPort()) {
    open.ResponsePort.Post(
      DefaultSubmitResponseType.Instance);
  }
  else {
    throw (new Exception(“Open Failed”));
  }
}

This handler calls OpenPort, an internal method. If this is successful, a response is posted back. Because no information has to be returned, this is just a default response provided by DSS.

If the open fails, then an exception is thrown. DSS catches the exception and converts it to a fault, which is sent back as the response. Though not immediately obvious, if an exception occurs in OpenPort, it will also bubble up and return a fault.

There’s no need to explain all of the OpenPort method, but one point is important—you can open the serial port for either synchronous or asynchronous operation. In synchronous mode, all read and write requests complete before returning a response. However, if you open in asynchronous mode, notifications are sent for every character received. To make this work, the code sets up a conventional event handler:

if (_state.Asynchronous) {
  // Set up an Event Handler for received characters
  sp.ReceivedBytesThreshold = 1;
  sp.DataReceived += new SerialDataReceivedEventHandler(
    DataReceivedHandler);
}

The event handler is shown in Figure 2. This handler runs on a .NET thread. But to interoperate with DSS, it must be switched to a CCR thread, so the code posts a ReceiveByte request to the service’s own main operations port. After posting a request, the code should receive the response, otherwise there will be a memory leak. This is the purpose of Arbiter.Choice, which uses the C# shorthand notation for anonymous delegates to handle the two possible responses. An Activate is necessary to place the Choice receiver into the active queue. Only a fault is relevant in this case, and a successful result does nothing.

Figure 2 Event Handler

void DataReceivedHandler(object sender, 
  SerialDataReceivedEventArgs e) {

  int data;
  while (sp.BytesToRead > 0) {
    // Read a byte - this will return immediately because
    // we know that there is data available
    data = sp.ReadByte();
    // Post the byte to our own main port so that
    // notifications will be sent out
    ReceiveByte rb = new ReceiveByte();
    rb.Body.Data = (byte)data;
    _mainPort.Post(rb);
    Activate(Arbiter.Choice(rb.ResponsePort,
      success => { },
      fault => { LogError("ReceiveByte failed"); }
      ));
  }
}

The ReceiveByte handler will be executed next to process the new character:

[ServiceHandler]
public void ReceiveByteHandler(ReceiveByte recv) {
  _state.LastByteReceived = recv.Body.Data;
  // Send a notification, but only if in asynch mode
  if (_state.Asynchronous)
    SendNotification(_submgrPort, recv);
  recv.ResponsePort.Post(DefaultUpdateResponseType.Instance);
}

The [ServiceHandler] attribute lets DSS hook the handler to the operations port during service initialization. The handler sends a notification to subscribers, then posts back a response saying the operation is complete. (Services that wish to receive notifications must send a Subscribe operation to the SerialPort service).

Service handlers for the other read and write operations are fairly self-explanatory. The WriteStringHandler (see Figure 3) contains one small wrinkle, though—it can insert a small delay between sending characters. This is designed to help slower microcontrollers that might not be able to keep up if the data is sent at full speed, 
especially devices like the BasicStamp that do bit banging and don’t have a hardware Universal Asynchronous Receiver/Transmitter (UART), so they perform serial I/O in software .

Figure 3 WriteStringHandler

[ServiceHandler]
public virtual IEnumerator<ITask> WriteStringHandler(
  WriteString write) {

  if (!_state.IsOpen) {
    throw (new Exception("Port not open"));
  }

  // Check the parameters - An empty string is valid, but not null
  if (write.Body.DataString == null)
    throw (new Exception("Invalid Parameters"));

  // NOTE: This might hang forever if the comms link is broken
  // and you have not set a timeout. On the other hand, if there
  // is a timeout then an exception will be raised which is
  // handled automatically by DSS and returned as a Fault.
  if (_state.Config.InterCharacterDelay > 0) {
    byte[] data = new byte[1];
    for (int i = 0; i < write.Body.DataString.Length; i++) {
      data[0] = (byte)write.Body.DataString[i];
      sp.Write(data, 0, 1);
      yield return Timeout(_state.Config.InterCharacterDelay);
    }
    sp.WriteLine("");
  }
  else
    sp.WriteLine(write.Body.DataString);

  // Send back an acknowledgement now that the data is sent
  write.ResponsePort.Post(DefaultSubmitResponseType.Instance);
  yield break;
}

Another point about this handler is that it is an Iterator. Notice the declaration of the method as IEnumerator<ITask> and the fact that it uses yield return and yield break. This feature of the C# language allows CCR to suspend tasks without blocking a thread. When the yield return is executed, the thread executing the method is returned to the pool. Once the timeout has completed, it posts back a message that causes execution to resume, though possibly on a different thread.

Configuring the Service

A configuration file is an XML serialized version of the service state that’s loaded during service initialization. It’s a good idea to support a config file because it allows you to change the behavior of the service without recompiling. This is especially important for setting the COM port number.

You can specify the name of the file in the service when you declare the state instance (in SerialPort.cs):

[ServiceState]
// Add an initial state partner to read the config file
[InitialStatePartner(Optional = true, 
  ServiceUri = "SerialPort.Config.xml")]
SerialPortState _state = new SerialPortState();

In this case we’ve declared the config as optional; otherwise, the service won’t start if the config file is missing. On the other hand, this means you have to check the state in the service’s Start method and initialize it to some sensible defaults if necessary.

Because no path is specified for the ServiceUri, DSS will assume that the file is in the same folder as the manifest used to start the service. Figure 4 shows the contents of a typical config file.

Figure 4 Config File for SerialPortState

<?xml version="1.0" encoding="utf-8"?>
<SerialPortState 
  xmlns:xsd="https://www.w3.org/2001/XMLSchema" 
  xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" 
  xmlns=https://www.promrds.com/contracts/2009/12/serialport.html
  >
  <OpenOnStart>false</OpenOnStart>
  <Asynchronous>false</Asynchronous>
  <Config>
    <PortNumber>1</PortNumber>
    <BaudRate>57600</BaudRate>
    <Parity>None</Parity>
    <DataBits>8</DataBits>
    <StopBits>One</StopBits>
    <ReadTimeout>0</ReadTimeout>
    <WriteTimeout>0</WriteTimeout>
    <InterCharacterDelay>0</InterCharacterDelay>
  </Config>
  <LastByteReceived>0</LastByteReceived>
  <IsOpen>false</IsOpen>
</SerialPortState>

Note that this config file does not request the service to open the COM port automatically, so you’ll have to send an Open request.

If you want your service to have a professional look, you need to pay attention to details such as the service description and icons. Even if you don’t plan to use VPL yourself, it’s a good idea to test your service in VPL and make it VPL-friendly so other people can use it easily.

You can decorate your service class with two attributes that describe the service:

[DisplayName("Serial Port Service")]
[Description("SerialPort service provides access to a Serial (COM) Port")]

The first attribute displays as the name of the service in the VPL service list (see Figure 5), and the second attribute appears as a tooltip if you mouse over the name of the service in the list.

Figure 5 VPL Service List

If you have a Web site and you want to document your services online, you can attach another attribute to the service class to provide the hyperlink:

[DssServiceDescription("https://www.promrds.com/SerialPort.htm")]

When VPL sees this attribute, it adds a small information icon (a white “i” on a blue disc) beside the service in the service list.

If you add icons to your service, which you can easily do, they’ll appear in VPL diagrams and the service list. Notice the small icon, called a thumbnail, beside the name of the service in Figure 5. Figure 6 illustrates the larger version of the icon, which will show up inside the block that appears in VPL diagrams.

Figure 6 Service Block

When you add these images to a project, make sure you change the file properties so that Build Action is set to Embedded Resource. PNG is the preferred file format because it supports an alpha channel. This  lets you create an icon with a transparent background by setting the alpha value on the background color to zero.

The service icon image should be 32 by 32 pixels and the thumbnail 16 by 16. The file names of the images must begin with the class name of the service, in this case SerialPortService. Thus I made the file names for my example SerialPortService.Image.png and SerialPortService.Thumbnail.png.

Using the Service

The service is quite flexible. You can specify the serial port configuration in a config file (which is the most common way) or by sending a SetConfig request. Once you’ve set the config, you can call the Open operation. For convenience, a flag in the config file will cause the service to open the port automatically on startup. If the port is already open, the Open call will first close, then re-open it.

You need to decide how you want to use the service: synchronously or asynchronously. When operating synchronously, each read or write request will wait until the operation completes before sending back a response. In terms of VPL, this is a simple approach, because the message flow will pause at the SerialPort block in the diagram. Note that asynchronous operation is not fully asynchronous—write operations still occur synchronously. But every new byte received is sent as a notification to subscribers.

In theory, every state-modifying operation on a service should cause a notification to be sent so that subscribers can keep their own cached version of the service state. This means all operations based on the DSSP Replace, Update, Insert, Delete and Upsert operations should send a corresponding notification. However, developers often play fast and loose with this requirement.

For simplicity, the SerialPort service only sends notifications using the ReceiveByte operation type when in asynchronous mode. The Open, Close, and SetConfig operations also cause notifications to be sent.

Because the Read and Write operations do not modify the state, they are subclassed off the DSSP Submit operation. Of course, they have a side-effect, which is to receive and send data over the serial link.

Testing with VPL

The download for this article includes two sample VPL programs, EchoAsynch and EchoSynch, which show how to use the service in both asynchronous mode (via notifications) and synchronous mode. The VPL samples use config files to set the initial parameters for the COM port, including the port number (which is set to 21 in the config files and must be changed to match your computer’s COM port address).

Note that to test the service you will need a null modem cable and either two computers with serial ports or two ports on the same computer. USB-to-serial devices are readily available, so it’s quite feasible to have multiple serial ports on a single PC. When you have your COM ports connected together, run a terminal emulator such as HyperTerminal and connect to one of the COM ports. Start by running the EchoAsynch VPL program on the other serial port. (You can find VPL in the Start menu under RDS). When you type into the terminal emulator window, you should see the characters echoed.

You can use VPL to create config files. Click on a SerialPort service block in the diagram and look in the Properties panel. You should see something like Figure 7. Make sure the PortNumber is set correctly for your PC. (This must be the other serial port, not the one you opened in the terminal emulator). You’ll notice that Parity and StopBits are drop-down lists. The entries in these lists come directly from the enums defined in SerialPortTypes.cs.

Figure 7 Serial Port Service Configuration

This is one place where XML doc comments come in handy. When you hover over a config parameter with the mouse cursor, a tooltip will pop up showing the corresponding comment for each state member.

When you run the EchoAsynch VPL program, the first part of the program in Figure 8 opens the serial port in asynchronous mode.

Figure 8 Opening the Serial Port in Asynchronous Mode

If you’ve entered invalid values in the config, such as a bad baud rate, the Open will fail. (It might also fail if the COM port is in use, the port doesn’t exist, or you don’t have appropriate permissions). This is why the program checks for a fault and displays it.

The rest of the program (see Figure 9) just echoes every character received. It does this by taking the characters from ReceiveByte notifications and sending them back using WriteByte.

To make the output easier to read, a carriage return character (ASCII 13, decimal) has a linefeed (10) appended so the cursor will move to the next line in your terminal emulator window. Note that all of the SerialPortService blocks in the Figure 9 diagram refer to the same instance of the service.

Figure 9 EchoAsynch Program

The EchoSynch VPL program (see Figure 10) uses the synchronous mode of operation—it doesn’t use notifications. (In fact, notifications are never sent in synchronous mode).

Figure 10 The EchoSynch Program

Unlike the previous program, this one uses ReadString and WriteString to echo data. These operations perform functions similar to those of ReadByteArray and WriteByteArray, However, strings are easier to handle in VPL than byte arrays.

The string operations use a linefeed (or newline) character as an end-of-line marker. So ReadString completes when you press linefeed (Ctrl-J) not when you press Enter. This can be confusing the first time you test the program using a terminal emulator, because you might wonder why nothing is echoing. WriteString adds a carriage return and linefeed to the output so each string appears on a separate line.

Note that the config file for EchoSynch has an InterCharacterDelay of 500ms. This causes the strings to be sent very slowly. You can try changing this value.

Trevor Taylor*, Ph.D., is the program manager for Robotics Developer Studio at Microsoft. He’s the coauthor, with Kyle Johns, of “Professional Robotics Developer Studio” (Wrox, 2008).*

* *

Thanks to the following technical experts for reviewing this article:Kyle Johns