Power to the Pen

The Pen is Mightier with GDI+ and the Tablet PC Real-Time Stylus

Charles Petzold

This article discusses:

  • InkOverlay versus the Real-Time Stylus
  • Implementing Real-Time Stylus plug-ins
  • Saving strokes and rendering with GDI+
  • Disappearing ink, packet transformations, and other fun activities
This article uses the following technologies:
Tablet PC, .NET Framework, C#

Code download available at:TabletPC.exe(170 KB)

Contents

What's Wrong with InkOverlay?
Implementing the Interface
Packets of Data
Converting the Points
Sync and Async
The Uniclass Real-Time Stylus
The Other Methods
Saving Strokes and Ink
The Pen Problem
Rendering with Windows Forms Pens
Stylus Pressure
Disappearing Ink
Chaining the Plug-Ins
The Stylus is a Method Call!

Many programming interfaces are the result of a compromise between the polar ideals of simplicity and versatility. Some, however, break the interface in two and pursue both goals independently. They keep the simple programming interface for many common programming tasks, but also provide a versatile interface for more unusual needs.

Such a separation between the simple and versatile characterizes the application programming interface for the Tablet PC. Much of the Tablet PC API is remarkably easy. With just a few lines of code you can attach an InkOverlay object to any Windows® Forms control and start doing some of the stuff Paul Yao demonstrated in his article "Add Support for Digital Ink to Your Windows Application" in the December 2004 issue of MSDN®Magazine.

Yet, as a result of this simplicity, InkOverlay has some severe and inescapable limitations. If InkOverlay doesn't quite meet your needs, you may want to make the leap to the other Tablet PC programming interface—the far more versatile Real-Time Stylus. That's what this article is all about.

What's Wrong with InkOverlay?

Generally, you attach an InkOverlay object to a control (or form) to perform two vital functions. First, InkOverlay renders stylus input on the screen using a pen described by a DrawingAttributes object. Second, the stylus input is accumulated in an Ink object as a collection of Stroke objects. (A single stroke occurs when the user touches the stylus to the screen, moves it, and then lifts it.) Ink objects can be saved to files in various formats, copied to the clipboard, or passed to a recognizer for conversion to text or gestures.

This is great stuff, but there are several limitations. Let's start with the pen that InkOverlay uses to render the strokes of the stylus. This is an object of type DrawingAttributes, and a cursory examination of the DrawingAttributes properties reveals that this is not the Pen object that Windows Forms programmers have known and loved for the past several years. The Windows Forms Pen class in the System.Drawing namespace (and part of a graphics interface sometimes called GDI+) lets you base pens on bitmapped images and gradient brushes. But the pen used by InkOverlay to render stylus input is the crushingly boring two-decades-old Windows API pen.

If you want the strokes of the stylus to be rendered with a System.Drawing.Pen object, and perhaps to base this pen on a bitmapped image or gradient brush, you need to use the Real-Time Stylus. Or you may want to restrict the rendered strokes of the stylus to a subset of the control. With InkOverlay, you can set a rectangular clipping area, but if you want to clip to any arbitrary region, again you need to use the Real-Time Stylus.

The InkOverlay conveniently draws the strokes as the user is moving the stylus, but your program may need to react to stylus movement in ways other than simple line and curve drawing. You can prevent InkOverlay from drawing entirely, but for efficient substitutions of the InkOverlay rendering, Real-Time Stylus is the better tool.

The InkOverlay object gives you a certain amount of information about what's going on with the stylus. The NewPackets event can tell you when the stylus is being dragged across the control, and the Stroke event notifies you when the stylus is lifted and the stroke has been completed. However, if you require more information about stylus activity, the InkOverlay can't help and you'll need to turn to the Real-Time Stylus.

The Real-Time Stylus provides notifications of stylus activity in a form similar to mouse events. However, instead of using MouseDown, MouseUp, and MouseMove events, you implement methods named StylusDown, StylusUp, and Packets. How you handle that stylus input—how you draw it to the screen and how you save it—is entirely up to you. There is no default rendering and nobody saves the data for you.

Implementing the Interface

While most of the Tablet PC classes are located in the Microsoft.Ink namespace, the Real-Time Stylus classes are relegated to the namespaces Microsoft.StylusInput and Microsoft.StylusInput.PluginData. Of primary importance are two interfaces defined in Microsoft.StylusInput. As you'll recall, a Microsoft ®.NET Framework interface looks a lot like a class in that it contains methods and properties, but these methods and properties have no bodies. A class "implements" the interface by providing code for all the methods and properties that are defined in the interface.

The two Real-Time Stylus interfaces are named IStylusSyncPlugin and IStylusAsyncPlugin, and they both define the same 15 methods and one property. To use the Real-Time Stylus in any nontrivial way, you must define a class that implements one (or both) of these interfaces, which means that you must create a class that provides code for these 15 methods and one property. That seems like an onerous task—it took me some time to accept the fact that I was actually being required to create such a class—until you realize that most of the methods can simply have empty bodies in the form of matching curly brackets.

A class that implements one or both of these interfaces is called a plug-in and, in this article, all my classes that implement these interfaces will end in the word Plugin. Figure 1 shows a simple plug-in class called SimpleRendererPlugin.

Figure 1 A Simple Real-Time Stylus Plug-In

using System; using System.Drawing; using System.Drawing.Drawing2D; using System.Windows.Forms; using Microsoft.StylusInput; using Microsoft.StylusInput.PluginData; class SimpleRendererPlugin : IStylusAsyncPlugin, IStylusSyncPlugin { Control ctrl; Point ptLast; // Constructor public SimpleRendererPlugin(Control ctrl) { this.ctrl = ctrl; } // Required property public DataInterestMask DataInterest { get { return DataInterestMask.StylusDown | DataInterestMask.Packets | DataInterestMask.StylusUp; } } // Implemented methods public void StylusDown(RealTimeStylus sender, StylusDownData data) { ptLast = new Point(data[0], data[1]); } public void Packets(RealTimeStylus sender, PacketsData data) { using (Graphics grfx = ctrl.CreateGraphics()) { grfx.PageUnit = GraphicsUnit.Millimeter; grfx.PageScale = 0.01f; grfx.SmoothingMode = SmoothingMode.AntiAlias; for (int i = 0; i < data.Count; i += data.PacketPropertyCount) { Point pt = new Point(data[i], data[i + 1]); grfx.DrawLine(SystemPens.ControlText, ptLast, pt); ptLast = pt; } } } public void StylusUp(RealTimeStylus sender, StylusUpData data) { using (Graphics grfx = ctrl.CreateGraphics()) { grfx.PageUnit = GraphicsUnit.Millimeter; grfx.PageScale = 0.01f; grfx.SmoothingMode = SmoothingMode.AntiAlias; grfx.DrawLine(SystemPens.ControlText, ptLast, new Point(data[0], data[1])); } } // Methods with empty bodies public void CustomStylusDataAdded(RealTimeStylus sender, CustomStylusData data) {} public void Error(RealTimeStylus sender, ErrorData data) {} public void InAirPackets(RealTimeStylus sender, InAirPacketsData data) {} public void RealTimeStylusDisabled(RealTimeStylus sender, RealTimeStylusDisabledData data) {} public void RealTimeStylusEnabled(RealTimeStylus sender, RealTimeStylusEnabledData data) {} public void StylusButtonDown(RealTimeStylus sender, StylusButtonDownData data) {} public void StylusButtonUp(RealTimeStylus sender, StylusButtonUpData data) {} public void StylusInRange(RealTimeStylus sender, StylusInRangeData data) {} public void StylusOutOfRange(RealTimeStylus sender, StylusOutOfRangeData data) {} public void SystemGesture(RealTimeStylus sender, SystemGestureData data) {} public void TabletAdded(RealTimeStylus sender, TabletAddedData data) {} public void TabletRemoved(RealTimeStylus sender, TabletRemovedData data) {} }

This plug-in includes code for the DataInterest property and the three most commonly implemented methods in these interfaces: StylusDown, Packets, and StylusUp. The other 12 methods are defined at the bottom of the class with empty bodies. In addition, I've given SimpleRenderPlugin a constructor. An explicit constructor is not required in plug-ins but is often useful.

Implementing the DataInterest property is essential: the property must return the members of the DataInterestMask enumeration corresponding to the methods of the interface that you're interested in; otherwise, the methods will never be called. The three methods that SimpleRenderPlugin provides are called when the stylus touches the screen, when it is moved, and when it is released.

In the downloadable source code for this article, available from the MSDN Magazine Web site, the SimpleRendererDemo project includes SimpleRenderPlugin.cs and SimpleRendererDemo.cs. All source code was developed under Visual Studio® .NET 2003 with the .NET Framework 1.1 and the Tablet PC SDK 1.7 installed. Projects were created using the Empty Project option and assigned references to the System, System.Drawing, System.Windows.Forms, and Microsoft.Ink assemblies, the last of which shows up in the Add Reference dialog box as "Microsoft Tablet PC API." It is possible to install the Tablet PC SDK 1.7 on a machine without a tablet, in which case the mouse mimics the stylus.

The SimpleRendererDemo class inherits from Form and includes the following code in its constructor:

RealTimeStylus rts = new RealTimeStylus(this); rts.SyncPluginCollection.Add(new SimpleRendererPlugin(this)); rts.Enabled = true;

The code first creates an object of type RealTimeStylus. Except in a very special case, the RealTimeStylus object must be associated with a particular control and only receives stylus input directed to that control. In this case, the RealTimeStylus is associated with the application form.

The second statement creates an object of type SimpleRendererPlugin (again with the form object as an argument to the constructor) and adds that object to the SyncPluginCollection property of RealTimeStylus. As the name implies, this SyncPluginCollection is a collection of objects that implement the IStylusSyncPlugin interface. Finally, the RealTimeStylus object is enabled and you're ready to get started. You can now use the Real-Time Stylus to draw on the form's client area.

Packets of Data

The StylusDown, Packets, and StylusUp methods in SimpleRendererPlugin are called when the stylus is touched to the screen over the form's client area, when it moves while touching the screen, and when it's lifted. The calls to Packets (and StylusUp) continue even if the stylus moves beyond the form's client area.

These three methods have arguments of type StylusDownData, PacketsData, and StylusUpData, respectively, but these three classes all derive from StylusDataBase and they are basically the same. (The name of the StylusDataBase class is a bit awkward—the name does not refer to a "database" for the stylus, but instead a base class for stylus data.) StylusDataBase implements an indexer of type int, so you can essentially treat the object as an array of integers. StylusDataBase also has a GetData method that returns this array of integers directly.

These integers are grouped into "packets." Each packet contains the X and Y coordinates of the stylus and some other general information, primarily stylus pressure. The StylusDown and StylusUp methods are accompanied by just one packet. The Packets method, as the name implies, can be accompanied by more than one. The total number of integers in the array is available from the Count property of StylusDataBase. The number of integers per packet is provided by PacketPropertyCount. The number of packets is therefore Count divided by PacketPropertyCount. The for loop in the Packets method in SimpleRendererPlugin shows one way to access these packets.

The first two integers of each packet are always the stylus X and Y coordinates relative to the upper-left corner of the control. Units are hundredths of a millimeter, a coordinate system that in the dark ages of the Win32® API was referred to as HIMETRIC.

For StylusDown, the SimpleRendererPlugin class simply saves the coordinates of the stylus as the ptLast field. The Packets method does the drawing. It first obtains a Graphics object for the control (that's why the class wanted a Control object in its constructor), then sets the PageUnit and PageScale properties so that the page units of the Graphics object agree with the tablet data coordinate system, and turns on antialiasing. For each point in the packet, a line is drawn using SystemPens.ControlText, which is a predefined pen based on the default foreground color of controls. The last point is again saved in ptLast.

The StylusUp method is similar to Packets except that only a final point is involved, and it doesn't need to be saved. Though I generally implement StylusUp out of a sense of symmetry, it can usually be ignored without anybody noticing.

Converting the Points

The screen resolution under Windows is generally set at 96 or 120 pixels per inch. The resolution of the Tablet PC packet data is 2540 units per inch—approximately 20 to 25 times finer. When I first started working with the Real-Time Stylus, I was concerned that many Packets calls would come through where subsequent points would correspond to the same pixel. Code that called DrawLine when there was essentially no line to draw seemed wasteful.

For that reason, in my early Real-Time Stylus coding, I often converted the packets to pixels and then checked whether the point was truly different from the last point before calling DrawLine. Recently, however, I've decided to let the Graphics object do this conversion for me. I am no longer very concerned about the potential problem here. I have examined some of the Packets calls and have determined that there really are not a lot of packets falling between the pixels.

Later in this article, I'll show you a program using a pen created from a bitmapped image to demonstrate how to convert the packets to pixels. The motivation there, however, resulted from an image-scaling problem rather than a superfluous-packet problem.

Sync and Async

The Real-Time Stylus can deliver packet data to your program either synchronously or asynchronously. The terms are relative to the tablet software rather than to your application, which at first may cause you some confusion. If you choose the asynchronous option, the calls to StylusDown, Packets, and StylusUp occur in the application user interface thread. With the synchronous option, these calls occur in the tablet thread.

How do you choose? First, the two interfaces defined in the Microsoft.StylusInput namespace are named IStylusSyncPlugin and IStylusAsyncPlugin. Because these interfaces implement the same 15 methods and one property, it's just as easy to derive a plug-in from both interfaces rather than just one. The plug-ins shown in this article derive from both interfaces to make it easier for me (and you) to experiment with them. However, some plug-ins can be defined to derive from only one interface to strongly suggest they should be used in a synchronous or asynchronous manner.

The RealTimeStylus class contains two properties named SyncPluginCollection and AsyncPluginCollection. These both work in the same way. Both properties expose collections that include Add and Remove methods. Both collections can accept multiple plug-ins, but a particular plug-in can only be a part of one collection.

The tablet system component delivers data packets to the RealTimeStylus object in the tablet thread. On that same thread, the RealTimeStylus then delivers the packets to each of the plug-ins in the synchronous collection in sequence. Once the synchronous plug-in objects are finished processing, the data is put into an internal queue and control is returned to the system. The application thread picks up the data from the queue and delivers the packets to all the plug-ins in the asynchronous collection in sequence.

As a general rule, you should use synchronous plug-ins only when an immediate response is required, such as during rendering. Other plug-ins are better off in the asynchronous collection. However, a little experimentation reveals that the synchronous collection is more forgiving of sluggish code. With fast code, each Packets call generally delivers only one packet. If slow code in the Packets method can't quite keep up at that pace, then multiple packets will be consolidated into single calls. This consolidation of packets does not occur with plug-ins in the asynchronous collection. (Then again, this consolidation only helps if your code has a per-Packets call problem rather than a per-packet problem.)

Note, too, that the calls to synchronous plug-ins are blocking calls—they block the tablet thread until the calls return from the plug-in objects. This directly affects the responsiveness of the stylus. Thus, a plug-in's design must be carefully thought through before making it synchronous, especially when using synchronization objects (such as semaphores and mutexes) in the plug-in's implementation.

The Microsoft.StylusInput namespace includes two predefined plug-ins. DynamicRenderer (which I'll make use of later in this article) is defined as implementing only the IStylusSyncPlugin interface, suggesting that the important job of rendering works best in response to user input. The GestureRecognizer plug-in implements both interfaces so it can be used in a more versatile manner.

The Uniclass Real-Time Stylus

When experimenting with the Real-Time Stylus, sometimes it's convenient to have a single class in a single source code file that does double duty as both the application form and the plug-in. This is fairly easy to accomplish because any class can inherit from another class (Form, for instance) and any number of interfaces:

class MyAppForm : Form, IStylusSyncPlugin, IStylusAsyncPlugin { ... }

To implement the interfaces, of course, the class must include the 15 methods and one property defined in the interfaces. The code in the constructor to create RealTimeStylus adds the form object to the plug-in collection simply by referring to the object as this:

RealTimeStylus rts = new RealTimeStylus(this); rts.SyncPluginCollection.Add(this); rts.Enabled = true;

Because the plug-in methods are part of the form class, they can simply make calls to CreateGraphics, for example, without prefacing it with a control object.

The Other Methods

Although StylusUp, Packets, and StylusDown are undoubtedly the three most popular methods of the Real-Time Stylus interfaces, some of the other twelve may also be useful when developing some applications.

TabletAdded and TabletRemoved signal when tablet hardware is added or removed from the system. (External tablets generally have USB interfaces.) RealTimeStylusEnabled and RealTimeStylusDisabled signal changes in the Enabled property of the RealTimeStylus object. StylusButtonDown and StylusButtonUp refer to the push button on the side of the stylus that is sometimes used to alter the meanings of strokes.

The tablet can detect the presence of the stylus even if it's not touching the screen. A call to the StylusInRange method indicates that the stylus has come close enough to the tablet to be detected. StylusOutOfRange signals when the stylus has been moved a sufficient distance away from the tablet. These calls occur based on the distance of the stylus from the tablet, regardless of the distance between the stylus and the control.

If the stylus enters the control's air space, then the plug-in will receive a profusion of calls to InAirPackets. These calls stop when the stylus is moved away from the control, and the cessation is also signaled by a call to StylusOutOfRange, even if the stylus is still close to the tablet.

Some movements of the stylus (either in the air or touching the tablet) result in calls to SystemGesture, and a member of the SystemGesture enumeration indicates the gesture. The most common in-air gestures are HoverEnter and HoverLeave. The most common gestures that involve touching the stylus to the tablet are Tap and DoubleTap.

An application can place custom data in the Real-Time Stylus queue by calling the AddCustomStylusDataToQueue method of RealTimeStylus. Plug-ins are notified of the presence of this data by a call to CustomStylusDataAdded. Finally, a plug-in can choose to receive a call to the Error method that signals when an exception is thrown by one of the plug-ins.

Saving Strokes and Ink

The SimpleRendererDemo program has a severe problem. If you draw on the window using the stylus and then, for example, minimize and restore the window, the stylus output will be gone. Nobody is saving the strokes. This is one of the things that makes the Real-Time Stylus so scary. The built-in amenities are gone. When you use the Real-Time Stylus, this is one of the responsibilities you have to assume.

How should strokes be saved? If you were to use InkOverlay as a model, you would store all the strokes in an instance of the Ink class. So, let's try to write a simple plug-in that also saves strokes in an object of type Ink. The SimpleInkSaverPlugin class is shown in Figure 2. The class creates an object of type Ink that is saved as a private field and exposed through a public property. The StylusDown method creates a new ArrayList object that it uses to store the first point. The Packets method stores subsequent points of the stroke in the ArrayList object. When the stroke is completed, the StylusUp method obtains the array of Point objects from ArrayList, and creates a Stroke object that is added to the Ink.

Figure 2 Plug-In That Saves Strokes

using System; using System.Collections; using System.Drawing; using Microsoft.Ink; using Microsoft.StylusInput; using Microsoft.StylusInput.PluginData; class SimpleInkSaverPlugin : IStylusAsyncPlugin, IStylusSyncPlugin { Ink ink = new Ink(); ArrayList arrlstStroke; // Property public Ink Ink { get { return ink; } } // Required property public DataInterestMask DataInterest { get { return DataInterestMask.StylusDown | DataInterestMask.Packets | DataInterestMask.StylusUp; } } // Implemented methods public void StylusDown(RealTimeStylus sender, StylusDownData data) { arrlstStroke = new ArrayList(); arrlstStroke.Add(new Point(data[0], data[1])); } public void Packets(RealTimeStylus sender, PacketsData data) { for (int i = 0; i < data.Count; i += data.PacketPropertyCount) arrlstStroke.Add(new Point(data[i], data[i + 1])); } public void StylusUp(RealTimeStylus sender, StylusUpData data) { arrlstStroke.Add(new Point(data[0], data[1])); Point[] apt = (Point[]) arrlstStroke.ToArray(typeof(Point)); ink.CreateStroke(apt); } // Methods with empty bodies ... }

The SimpleInkSaverDemo project included with the downloadable code includes SimpleInkSaverPlugin.cs, a link to SimpleRendererPlugin.cs, and SimpleInkSaverDemo.cs, which has a class that derives from Form and creates a button, label, and panel on the surface of its client area. It creates a RealTimeStylus object based on the Panel object named pnl:

RealTimeStylus rts = new RealTimeStylus(pnl); rts.SyncPluginCollection.Add(new SimpleRendererPlugin(pnl)); inkplugin = new SimpleInkSaverPlugin(); rts.AsyncPluginCollection.Add(inkplugin); rts.Enabled = true;

Notice that the renderer plug-in is added to the synchronous plug-in collection while the ink-saver plug-in (saved as a field named inkplugin) is added to the asynchronous plug-in collection because it doesn't have to respond as quickly as the renderer.

The SimpleInkSaverDemo class makes use of the Ink object in two ways. First, an event handler for the panel's Paint event accesses the Ink object to repaint its surface:

foreach (Stroke stk in inkplugin.Ink.Strokes) grfx.DrawLines(SystemPens.ControlText, stk.GetPoints());

Secondly, clicking the button on the form causes the program to pass the Ink object to a recognizer that converts it into text to be displayed in the label:

RecognizerContext recoContext = new RecognizerContext(); recoContext.Strokes = inkplugin.Ink.Strokes; RecognitionStatus recoStatus; RecognitionResult recoResult = recoContext.Recognize(out recoStatus); lblOutput.Text = recoResult.TopString;

Figure 3** Ink Recognition Without InkOverlay **

Figure 3 shows some handwriting converted to text by the program. Yes, there can be understandable anxiety when first approaching the Real-Time Stylus because it doesn't implement Ink collection or recognition. I hope this example makes it clear that it's not very hard to add simple implementations of these features.

The Pen Problem

The real problem involved in creating Ink from a Real-Time Stylus application is the pen. When you're using InkOverlay, each Stroke object stored with the Ink has its own DrawingAttributes object that indicates (among other things) the color and width of the pen used to draw the stroke. If the strokes are to be properly redrawn on the screen from the Ink object, the DrawingAttributes object must be set to the same values used when originally rendering the stroke, and they must be taken into account in the repainting logic.

However, one big reason to use the Real-Time Stylus is to render with a GDI+ Pen object. Certainly the color and width of the GDI+ Pen could be stored in a DrawingAttributes object, but what if the pen is based on a hatch brush? One approach might be to save the Pen characteristics in the ExtendedProperties property of DrawingAttributes.

The SimpleInkSaverDemo program attempts to solve this problem by using SystemPens.ControlText both in the renderer plug-in and when redrawing the panel. However, that's certainly not a good general solution, and it can't be used in the other programs discussed in this article. Ultimately, the solution to the problem of reconciling Ink objects and Pen objects will need to be approached on an application basis: just how fancy is your Pen and how much detail about it do you need stored with each Stroke object?

Meanwhile, after knowing that it's possible to accumulate Ink and pass it to a recognizer, you can relax and have some fun. In the remainder of this article I'm going to focus on interesting approaches to rendering rather than saving and recognizing Ink.

Rendering with Windows Forms Pens

The GdiPlusRendererPlugin class included with this article's downloadable code is a renderer that uses a GDI+ Pen object settable through a public property of the plug-in. The plug-in creates a default pen in its constructor based on the foreground color of the control (or form) it's drawing on, and with a width of 53. This may seem like a strange number, but keep in mind that the pen width is in world coordinates, and when used with a Graphics object set for hundredths of millimeters, it's about half a millimeter. It's the same width you'll find in default DrawingAttributes objects and it was chosen because it is equivalent to 1.5 points. Most importantly, it looks about right.

When the plug-in receives a call to StylusDown, it saves the stylus point (as usual) but also clones the pen to use in subsequent drawing, as shown here:

ptLast = new Point(data[0], data[1]); pnStroke = (Pen) Pen.Clone();

This code ensures that the same pen is used for the entire stroke. The GdiPlusRendererPlugin class also includes a public property named Clip of type Region. The RenderData method (called from both Packets and StylusUp) uses this property to set a clipping region for the Graphics object:

if (Clip != null) grfx.Clip = Clip;

Figure 4 Flat Line Caps

Figure 4** Flat Line Caps **

The GdiPlusRendererDemo project includes GdiPlusRendererPlugin.cs and GdiPlusRendererDemo.cs. The GdiPlusRendererDemo class inherits from Form and creates the RealTimeStylus object, as usual. But it also creates a modeless dialog box containing a PropertyGrid control that lets you change the Pen properties interactively.

You can change the pen color, of course, and you'll want to try changing the pen width, as well. Keep in mind that the width is in hundredths of millimeters. A value of 635 is equivalent to one quarter inch. At widths of that size, you'll see something very disturbing happen, as shown in Figure 4.

The problem here is that GDI+ pens begin and end abruptly at the geometric start point and end point of the line. As you draw with the stylus, each piece of the line becomes just a sliver. Fixing this problem is fairly easy: change the StartCap and EndCap properties of the Pen to something other than the default LineCap.Flat. A value of LineCap.Round results in a filled half-circle added onto the ends and works well regardless of how sharp you draw corners, as Figure 5 shows.

Figure 5 Rounded Line Caps

Figure 5** Rounded Line Caps **

One of the powerful aspects of GDI+ pens is that they can be based on brushes. These can be solid brushes (SolidBrush), gradient brushes (objects of type LinearGradientBrush or PathGradientBrush), hatch brushes (HatchBrush), or brushes based on bitmaps or metafiles (TextureBrush). When you draw with the latter type, it's as if the pen is cutting a swath through a mask to reveal the brush-covered surface underneath. For example, you might create a gradient brush large enough to cover the client area of a form, with red at the right of the form and blue on the left. If you then base a pen on that brush, lines you draw at the right of the form will be red, and lines at the left will be blue. Lines in the middle will be magenta, and horizontal lines across the form will show the gradient.

The HatchBrushDemo class inherits from GdiPlusRendererDemo and creates a one-half inch-wide pen based on HatchStyle.HorizontalBrick. The resulting pen is set to the Pen property of the GdiPlusRendererPlugin object:

HatchBrush brsh = new HatchBrush(HatchStyle.HorizontalBrick, Color.White, Color.Black); gdiplus.Pen = new Pen(brsh, 1270); gdiplus.Pen.StartCap = gdiplus.Pen.EndCap = LineCap.Round;

This program also demonstrates the Clip property of GdiPlusRendererPlugin. Some simple code creates a five-pointed star in an array of five PointF structures named aptf. From these points, a closed path is created:

GraphicsPath path = new GraphicsPath(); path.AddLines(aptf); path.CloseFigure();

When converted to a region, that path becomes the clipping region:

gdiplus.Clip = new Region(path);

The stylus in this program draws bricks clipped to the interior of the five-pointed star, as shown in Figure 6. In a similar spirit, the ImagePenDemo program (which consists of the classes ImagePenDemo and ImagePenPlugin) creates a TextureBrush based on a bitmap or metafile and uses that brush to create the pen.

Figure 6 Stylus Output with Hatch Pen and Clipping

Figure 6** Stylus Output with Hatch Pen and Clipping **

Windows Forms is usually quite good about using embedded resolution information in bitmaps to display images at their proper size. But that doesn't seem to be the case for texture brushes. When using a Pen based on a TextureBrush and drawing with page units of hundredths of a millimeter, the bitmap image shrinks in size. This problem can be fixed by setting the Transform property of the TextureBrush object to scale up the size of the image, but I decided it might be useful showing an example where drawing is done in units of pixels.

The ImagePenPlugin class has two public properties named Image and WidthInPoints (my preferred device-independent units). It creates a pen during the StylusDown method:

TextureBrush tb = new TextureBrush(Image, WrapMode.Tile); float fPenWidth = grfx.DpiX * WidthInPoints / 72; pn = new Pen(tb, fPenWidth); pn.StartCap = pn.EndCap = LineCap.Round;

The RenderData method (called from both Packets and StylusUp) converts packet points to pixels and draws a line based on this pen, as shown in the following:

Point pt = new Point((int) (grfx.DpiX * data[i + 0] / 2540 + 0.5), (int) (grfx.DpiY * data[i + 1] / 2540 + 0.5)); if (pt != ptLast) { grfx.DrawLine(pn, ptLast, pt); ptLast = pt; }

The ImagePenDemo class creates the RealTimeStylus object and adds the ImagePenPlugin object to its SyncPluginCollection. The class also creates a modeless dialog box with a PropertyGrid control for the ImagePenPlugin class, letting the user set the Image and WidthInPoints properties directly. (Using the PropertyGrid to set a property of type Image is an easy way to get a File Open dialog box up on the screen.) Once an image is selected and then loaded, each stroke of the stylus reveals more of the image, as you can see in Figure 7.

Figure 7 Stylus Output Using ImagePen

Figure 7** Stylus Output Using ImagePen **

Stylus Pressure

The programs shown so far in this article have used only the first two integers of each packet. The third integer, if it exists, always indicates stylus pressure ranging from 0 to 255. The DrawingAttributes object used with InkOverlay and Ink includes a property named IgnorePressure that is normally set to false. By default, the pressure of the stylus on the screen causes the width of the rendered line to vary between 50 percent and 150 percent of its normal width.

Mimicking this behavior is fairly easy, but it's more fun to take it a step further and accentuate the effect. The StylusPressureWidthDemo program (consisting of StylusPressureColorPlugin.cs and StylusPressureWidthDemo.cs) contains the following code that creates a new pen for each packet:

float fWidth = 53; if (data.PacketPropertyCount > 2) fWidth *= 1 + (data[i + 2] - 128f) / 16; Pen pn = new Pen(ctrl.ForeColor, fWidth);

To reproduce the normal variation associated with stylus pressure, change the denominator in the fWidth calculation to 255.

It's also possible to reflect pressure width in other ways. The StylusPressureColorDemo program (consisting of StylusPressureColorPlugin.cs and StylusPressureColorDemo.cs) varies the pen color from blue to red depending on the stylus pressure:

Color clr = ctrl.ForeColor; if (data.PacketPropertyCount > 2) clr = Color.FromArgb(data[i+2], 0, 255-data[i+2]); Pen pn = new Pen(clr, 53);

Another possibility for use with a white background is to vary the pen color from white to black based on the pressure:

int iShade = 255 – data[i + 2]; clr = Color.FromArgb(iShade, iShade, iShade);

Disappearing Ink

Entirely invisible ink is a snap, even if you're using InkOverlay. Simply set the RasterOperation property of the DrawingAttributes to the enumeration value RasterOperation.NoOperation. Somewhat more of a challenge is to produce ink that draws normally but fades out over time.

I originally wrote the DisappearingInk program simply for my own amusement, but somebody pointed out to me that this would be a great feature in a note-taking program, particularly when you're working in public and want to hide what you're writing from roving eyes. You get the immediate feedback of ink from the stylus, but then it fades out and can't be seen.

The DisappearingInk program begins with a class named TimeStampedPoint. This class, shown in Figure 8, simply combines a Point structure and a DateTime structure. The read-only property Age indicates the age of the object in milliseconds.

Figure 8 A Point That Reveals Its Age

using System; using System.Drawing; class TimeStampedPoint { Point pt; DateTime dt; public TimeStampedPoint(Point pt) { this.pt = pt; dt = DateTime.Now; } public Point Point { get { return pt; } } public long Age { get { return (long) (DateTime.Now - dt).TotalMilliseconds; } } }

The DisappearingInkPlugin class maintains two ArrayList objects: alPoints and alStrokes. The first ArrayList is a collection of TimeStampedPoint objects that make up the current stroke. The StylusDown method creates a new alPoints object to signal the beginning of a new stroke. The method also adds that alPoints object to alStrokes—the ArrayList that stores all the strokes. The Packets and StylusUp calls draw the packets normally, but they also add additional TimeStampedPoint objects to alPoints.

The constructor of DisappearingInkPlugin starts a timer. The Tick event handler steps through all the points of all the strokes, and redraws the lines using a color calculated based on the Age property of each point.

Doing strange things with the Real-Time Stylus can become an obsession. On example: my quest to display drop-shadow on stylus input is documented on my Web site at In Search of the Real-Time Drop Shadow.

Chaining the Plug-Ins

As you saw in the SimpleInkSaver program, it's possible to have a chain of multiple plug-ins. This is sometimes useful in dividing the processing of stylus input, perhaps between rendering and saving. But it's also possible for plug-ins earlier in the chain to alter the data seen by plug-ins later in the chain.

If a plug-in wants to change the packet data but retain the same number of data packets, it can simply change all or some of the integer values stored in the data object passed to the StylusDown, Packets, or StylusUp methods. If the plug-in needs to change the number of packets, it calls the SetData method defined by StylusDataBase with the new array of integers. If the SetData method is called with a null argument, then the event is canceled for all subsequent plug-ins.

The remaining two example programs in this article use DynamicRenderer for rendering the strokes. DynamicRenderer is a class included in the Microsoft.StylusInput namespace that implements IStylusSyncPlugin and renders stylus input to the screen based on a DrawingAttributes object.

The DynamicRenderer constructor requires a Control object, and the Enabled property must be set to true. If you also set the EnableDataCache property to true, then DynamicRenderer retains all the strokes and can repaint the control with a call to its Refresh method. However, DynamicRenderer does not expose this collection of strokes so it can't be used to fabricate an Ink object. It doesn't make much sense to use DynamicRenderer by itself; it's more useful in conjunction with other plug-ins. Although the next two programs use the DynamicRenderer constructor for drawing the strokes, they use custom plug-ins for altering the packet data before it gets to DynamicRenderer.

The RotateAroundStart project includes the RotateAroundStartPlugin class (most of which is shown in Figure 9) and the RotateAroundStart class. The plug-in has a constructor that requires an angle (in degrees) as an argument. This is saved in a field. In the StylusDown method, a matrix transform is created that defines a rotation around the point at which the stylus touched the screen:

xform = new Matrix(); xform.RotateAt(fAngle, new Point(data[0], data[1]));

This transform is saved as a field. Both the Packets and StylusUp methods call the TransformPoints method. This method begins by transferring all the packet points to a Point array. It then uses the Matrix object to rotate the points, and stores the transformed points back to the StylusDataBase object.

Figure 9 Transforming Packet

using System; using System.Drawing; using System.Drawing.Drawing2D; using System.Drawing.Imaging; using System.Windows.Forms; using Microsoft.StylusInput; using Microsoft.StylusInput.PluginData; class RotateAroundStartPlugin : IStylusSyncPlugin, IStylusAsyncPlugin { float fAngle; Matrix xform; // Constructor public RotateAroundStartPlugin(float fAngle) { this.fAngle = fAngle; } // Required property public DataInterestMask DataInterest { get { return DataInterestMask.StylusDown | DataInterestMask.Packets | DataInterestMask.StylusUp; } } // Implemented methods. public void StylusDown(RealTimeStylus sender, StylusDownData data) { // Create matrix for rotation around point. xform = new Matrix(); xform.RotateAt(fAngle, new Point(data[0], data[1])); } public void Packets(RealTimeStylus sender, PacketsData data) { TransformPoints(data); } public void StylusUp(RealTimeStylus sender, StylusUpData data) { TransformPoints(data); } void TransformPoints(StylusDataBase data) { Point[] apt = new Point[data.Count / data.PacketPropertyCount]; // Get the points. for (int i = 0; i < data.Count; i += data.PacketPropertyCount) apt[i] = new Point(data[i], data[i + 1]); // Transform the points. xform.TransformPoints(apt); // Set the points. for (int i = 0; i < data.Count; i += data.PacketPropertyCount) { data[i] = apt[i].X; data[i + 1] = apt[i].Y; } } // Methods with empty bodies ... }

Now let's see how this class is used. RotateAroundStart inherits from Form and defines iAngle as 30 degrees. After creating a RealTimeStylus object, the program adds to the SyncPluginCollection no fewer than 12 instances of RotateAroundStartPlugIn and 12 instances of DynamicRenderer in an alternating pattern:

for (int i = 0; i < 360 / iAngle; i++) { rts.SyncPluginCollection.Add(new RotateAroundStartPlugin(iAngle)); DynamicRenderer dynaren = new DynamicRenderer(this); dynaren.Enabled = true; rts.SyncPluginCollection.Add(dynaren); }

Notice that the argument to the RotateAroundStartPlugin constructor is always the same iAngle value. The first instance rotates the packet points 30 degrees, the second instance 30 more degrees, and so forth. The result is a pattern centered around the initial point. Figure 10 shows one of many possible results. But watch out—by installing 24 plug-ins in a chain, RotateAroundStart is certainly pushing the envelope of the capabilities of the Real-Time Stylus. If you draw very quickly, some packets will be dropped and it won't be quite as symmetrical.

Figure 10 Twelve Transforms and Renderings

Figure 10** Twelve Transforms and Renderings **

RotateAroundStart overrides the OnDoubleClick method of the form and calls Invalidate to erase the client area. In such a fun program, that seemed more useful than saving and restoring the strokes.

The Whirligig program installs just one instance of WhirligigPlugin and one instance of DynamicRenderer, but it replaces the entire data array during the Packets and StylusUp calls to display a spiral, as shown in Figure 11.

Figure 11 Whirligig Spirals Around Stylus Tip

Figure 11** Whirligig Spirals Around Stylus Tip **

For each movement of the stylus, Packets and StylusUp both call a method named Generate that calculates points constituting an arc at a constant distance from the stylus point. The mathematics was a little tricky, and filling up the array of integers to pass to SetData also required a bit of thought.

The PacketPropertyCount is read-only and cannot be changed. The Count property is also read-only, but it will reflect the new number of integers specified in the SetData call. The number of integers in the array must be PacketPropertyCount times the number of new packets.

WhirligigPlugIn defines the ArrayList object named arrlst to store the new points. The Count property of the array list is the number of Point structures it stores, and hence also the number of packets. The array of integers must have PacketPropertyCount items for each Point in the ArrayList:

int[] arr = new int[data.PacketPropertyCount * arrlst.Count];

The Point coordinates are stored in the first two elements of each packet as shown here:

for (int i = 0, j = 0; i < arr.Length; i += data.PacketPropertyCount) { Point pt = (Point) arrlst[j++]; arr[i] = pt.X; arr[i + 1] = pt.Y;

The remainder of the packet must also be set. I simply used the additional integers from the first original packet:

for (int k = 2; k < data.PacketPropertyCount; k++) arr[i + k] = data[k]; }

In most cases, PacketPropertyCount will be 3 and that third integer will be the stylus pressure. DynamicRenderer responds to that pressure, so it's important to pass it through.

Finally, calling SetData replaces the original array of points in the data object with the following array:

data.SetData(arr);

This new array of points is the data that the DynamicRenderer will see and render.

The Whirligig class sets the EnableDataCache property of the DynamicRenderer to true to save all the strokes. These are, of course, the modified strokes created by WhirligigPlugin. A call to Refresh causes DynamicRenderer to redraw these strokes, and a good place to call Refresh is in the control's OnPaint method. For that reason, Whirligig overrides the OnPaint method of the form.

The documentation of Refresh reads "When calling the DynamicRenderer object's Refresh method with a Paint event handler, set the DynamicRenderer object's ClipRectangle property to the PaintEventArgs object's ClipRectangle property." But it doesn't really mean that literally. The documentation really means to say that you might temporarily set the ClipRectangle property of the DynamicRenderer object to save some processing time. If you don't restore the ClipRectangle of DynamicRenderer, then subsequent stylus input will be restricted to that rectangle. Figure 12 shows how the OnPaint method of Whirligig sets the ClipRectangle of the DynamicRenderer object to refresh the form's client area but then restores it.

Figure 12 Refreshing the Client Area Using DynamicRenderer

using System; using System.Drawing; using System.Windows.Forms; using Microsoft.StylusInput; class Whirligig : Form { DynamicRenderer dynaren; public static void Main() { Application.Run(new Whirligig()); } public Whirligig() { Text = "Whirligig"; RealTimeStylus rts = new RealTimeStylus(this); rts.SyncPluginCollection.Add(new WhirligigPlugin()); dynaren = new DynamicRenderer(this); dynaren.Enabled = true; dynaren.EnableDataCache = true; rts.SyncPluginCollection.Add(dynaren); rts.Enabled = true; } protected override void OnPaint(PaintEventArgs args) { Rectangle rectSave = dynaren.ClipRectangle; dynaren.ClipRectangle = args.ClipRectangle; dynaren.Refresh(); dynaren.ClipRectangle = rectSave; } }

The Stylus is a Method Call!

The stylus of the Tablet PC looks like a pen, so it's natural to render the strokes of the stylus with lines that resemble ink or pencil or a felt-tip marker. And it's helpful that InkOverlay renders stylus input in a format that is familiar and comforting to users. But the stylus is not a real pen or a pencil or a felt-tip marker. Ultimately, the stylus is a series of method calls, and your program can respond to these method calls in whatever way you see fit. Breaking free of the restraints of InkOverlay into the realm of the Real-Time Stylus is the necessary first step.

Charles Petzold, a long-time contributing editor to MSDN Magazine and its predecessor, MSJ, is the author of four books on .NET Framework programming, and a forthcoming book on the Windows Presentation Foundation (all from Microsoft Press). His Web site is www.charlespetzold.com.