Capture and Playback of Ink Data in Tablet PC Applications


Todd Landstad
Microsoft Corporation

January 2006

Applies to:
   Microsoft Windows XP Tablet PC Edition 2005
   Microsoft Windows XP Tablet PC Edition Development Kit (SDK)

   Microsoft Windows XP Tablet PC Edition SDK
   Microsoft Windows Journal Reader Supplemental Component for the Windows XP Tablet PC Edition Development Kit 1.7

Summary: This article describes how to capture ink data and later use that data to automate the testing of ink-based applications or controls. The corresponding sample is written in C#. (7 printed pages)

Click here to download the sample code for this article.


The Impact of Ink Capture and Playback
Using the Samples


Ink Recorder is a mechanism for capturing and playing back information associated with data collected from Ink objects. The Ink Recorder sample enables some new scenarios for writing, debugging, and testing ink-enabled applications. Whether you are interested in creating unit tests for an ink-enabled control, creating automated tests to validate stability in your application, or adding some debugging ability for ink, the InkRecorder object's capture and playback functionality may help you.

While capturing data for future playback is not a new concept (consider the macro recorder in Windows), the Ink Recorder exposes some useful concepts for ink-enabled applications that were not previously available. Ink Recorder takes full advantage of the detail of the ink capabilities within the Tablet PC Platform API.

This article explores new concepts such as automating ink creation for testing and debugging new applications, or even as a unique feature in new ink based applications. It then describes the sample code that features these new concepts.

The Impact of Ink Capture and Playback

You can implement the Ink Recorder sample to improve your Tablet PC-based application, specifically as:

  • A testing tool. Reap the benefits of Automation Testing by using ink control testing.
  • A debug application feature. Load persisted documents and play them back as they were created.
  • An application feature. Add real-time playback to ink drawing, for example.

Consider the following benefits to using automated tests while writing ink-enabled applications or controls

Ink Recorder as a Testing Tool

When they fail, unit tests give a quick notification of any regressions in your code. For ink based controls or applications, user input may play a critical part in testing. Being able to create ink by using the same means as a user turns a manual testing scenario into a deterministic automated test. Ink Recorder can be of prime importance in creating these controlled automated tests. Additionally, you can improve the efficiency of testing any feature changes. Automating the process of verifying enabled Ink objects or recognition ability across the supported scenarios or even platforms (C# or C++, for example) in your application greatly reduces the time you need to spend on testing those changes.

Ink Recorder as a Debug Application Feature

Use Ink Recorder to enable detailed debugging in your application—especially for beta-level applications—when you load and save ink. The accompanying sample supports Ink Serialized Format (ISF) and the default Journal note format (.jnt), but you are not limited to any format when you playback ink. In this scenario, when you application hits an issue it can save off the ink content at that moment. Using Ink Recorder, you can then play back that content as the user created it in order to attempt to reproduce the issue.

Ink Recorder as an Application Feature

While examining the samples that accompany this document, you may discover the possibility of creating specific application features around capturing and playing back ink data. There are several possible features that this play back could add value to—two possible examples are:

  • An instructional application that animates the creation of a painting, sketch or diagram. The content would literally be created as the viewer watches.
  • An advanced note-taking scenario where the user has recorded audio while taking notes. You can then synchronize the audio playback to the individual packets of ink so that as the audio track is played back, the viewer sees the exact ink strokes being created at that moment.

Using the Samples

In order to compile the sample source code, you must have Microsoft Visual Studio .NET 2003, Microsoft Tablet PC SDK 1.7 and the Journal Reader Supplemental Component for the 1.7 Tablet PC SDK installed.

To run these sample applications within Visual Studio, make sure you set the desired project as the StartUp Project. To do so, in Solution Explorer select the desired startup project. Click the Project menu, and then click Set as StartUp Project.

The InkRecorder solution contains two projects:

  • InkRecorderPlugin—A C# object that enables your managed application to capture ink data from the RealTimeStylus object and save that data to a file. In addition you can load ink from ISF or .jnt files and then play the data back to a specified window.
  • InkRecorderSample—A Windows application that instantiates the InkRecorder object and exposes menu items to control how the InkRecorder captures, saves, loads, and plays back ink.

The sample application can easily capture ink data as the user generates it, but you can also load existing ink files in both ISF and JNT formats:

Figure 1. Load ISF or Journal note files.

Figure 2. The sample application illustrates how to use the InkRecorder.

InkRecorderPlugin DLL: The Core C# Object

The InkRecorderPlugin class is the managed version of the Ink Recorder. InkRecorderPlugin has a small interface which provides the basic functionality of capturing, saving, loading, and playback of ink data.

A RealTimeStylus is optionally specified in the constructor of this object. If a RealTimeStylus is specified, you can capture user ink input as an available option.

InkRecorderPlugin has the following Interface:

    public class InkRecorder
        public void RecordStylusInput()
        public void RecordStylusInput(int duration)
        public void Stop()
        public void PlaybackRecordedData(float speedMultiplier)
        public void SaveRecordedData(string isfFileName)
        public void LoadRecordedData(string isfFileName)
        public void ClearRecordedDataCache()
        public InkRecorder(RealTimeStylus rts)

The following code illustrates how to use the Ink Recorder:

// Declare the InkRecorder
      private InkRecorder myInkRecorder;
   // Create the InkRecorder used to replay ink collection
      myInkRecorder = new InkRecorder(myRealTimeStylus);
   // Start recording ink
   // Save the recorded Ink data
   // Clear ink data from InkRecorder object
   // Load the file in the InkRecorder

   // Start the playback of recorded ink
   myInkRecorder.PlaybackRecordedData(myInkingPanel, myRenderer, 
      RespectAspectRatio, playbackRate);

InkRecorderSample: InkRecorder in Action

This sample application illustrates the use of the InkRecorder class. All the features of the InkRecorder are exposed, and you can use them without alteration to capture ink data. You can also load ISF or JNT files and play back the packet data to the inking area of the sample application. Each menu on the form of the sample coincides with one of the class features:

  • Use the File menu to open an ISF or .jnt file for playback to the sample window or save collected ink to an ISF file.
  • Use the Ink menu to change the drawing attributes of collected ink.
  • Use the Clear All Ink menu to remove the current ink from the window and clear the InkRecorder object's collected ink.
  • Use the Record Ink Data menu to notify the InkRecorder object to collect all ink that the user creates in the sample application's ink-collecting window.
  • Use the Playback Ink Data menu to notify the InkRecorder object to playback currently loaded ink to the sample application's ink-collecting window.
  • Use the Options menu to alter the playback speed change the aspect ratio of ink playback. If Keep Aspect Ratio is not selected, playback scales to fit the size of the playback window.


Automating user input for ink based applications can help improve quality, help to speed up development and potentially be a building block for new features in applications.

  • Simple Interface—The Ink Recorder sample provides a small, clean interface to capture and playback ink data. Can be modified to fit your application scenario.
  • Improved testing and debugging scenarios—You can play back ink content as the user created it in order to test inking features or to reproduce/debug reported issues
  • New potential ink features—Instructional-based applications may add a new dimension when playing back paintings, notes, or sketches as they were created.