Export (0) Print
Expand All
Add Support for Digital Ink to Your Windows Applications
Power to the Pen: The Pen is Mightier with GDI+ and the Tablet PC Real-Time Stylus
Expand Minimize
1 out of 2 rated this helpful - Rate this topic

Get the Best Results from Tablet PC Handwriting Recognizers

 

Peter Gruenbaum
Microsoft Corporation

June 2004

Applies to:
   Microsoft® Tablet PC Platform SDK
   Ink
   Handwriting Recognition

Summary: Describes how best to use handwriting recognizers so that they provide the best results for your application. These descriptions and examples—in C# and Microsoft Visual Basic .NET — use the Microsoft Tablet PC Platform SDK version 1.7 API. Readers should be familiar with the Microsoft Tablet PC Platform SDK and managed code. (21 printed pages)

Contents

Introduction
Providing Contextual Hints
Providing Location and Size Hints
Providing a User Interface to Select Alternates
Freeform Applications
Conclusions

Introduction

One useful aspect of a Tablet PC is that it can turn handwriting into text. When you are developing Tablet PC applications, you can take steps to ensure that handwriting recognition returns the best possible results. Contextual hints let the recognizer know what kind of result you are expecting, and hints about ink location and size let the recognizer know where you expect the user to be writing. Once you have performed recognition, you can display not only the recognizer's best guess, but other alternate possibilities that the recognizer has chosen.

This article describes how to provide contextual hints and hints about location, how to display recognition alternates, and how you could create an application where the user can use ink in a freeform manner. The article explains how to work with the Recognizer and RecognizerContext objects to achieve the best possible recognition results.

Sample code is included in C# and Visual Basic® .NET. To run the samples, you must have a computer that has recognizers installed, such as a computer running Microsoft Windows® XP Tablet PC Edition, and the Microsoft Tablet PC Platform SDK version 1.7. The sample code demonstrates most of the concepts described in this article. To use the sample application, write some ink words in the white panel. Click the check boxes to affect the RecognizerContext object, and then click Recognize to perform recognition on the ink.

This article discusses how to do the following:

  • Specify what language to use for recognition.
  • Provide contextual hints called input scopes (or factoids).
  • Limit the recognizer to look for a single word.
  • Use a word list for words that are not in the standard dictionary.
  • Provide hints about location and size to improve recognition.
  • Make use of recognition alternates.
  • Limit recognition alternates to those with the same segmentation as the top alternate.
  • Use character Autocomplete with recognizers of East Asian characters.

This article does not discuss the following:

  • How to create your own recognizer.
  • Gesture recognition.

Providing Contextual Hints

You can provide hints to the recognizer about what kinds of words it is expected to recognize. By providing this kind of context, you can increase the chances that the recognizer comes up with the interpretation that the user is expecting. Contextual hints include language, factoids, word mode, and word lists.

Language

Recognizers are available in several languages, both using the Latin alphabet and East Asian characters. In general, you use recognizers of Latin script and of East Asian characters in the same way, but there are a few exceptions. For most recognizers, you'll find the following differences:

  • Recognizers of East Asian characters support character Autocomplete, whereas recognizers of Latin script do not.
  • Generally recognizers of East Asian characters do not add spaces between characters. (The Korean recognizer is an exception.)
  • Recognizers of Latin script support WordList objects (see the Word List section that follows), but recognizers of East Asian characters do not.

To find a recognizer for a specific language, you can use the default recognizer if you know its National Language Support (NLS) Language Code Identifier (LCID), or you can loop through the Recognizers collection and find those whose Languages property contains the identifier for the language that you want to recognize. A table of LCIDs can be found at the Table of Language Identifiers.

Note   The Recognizers.GetDefaultRecognizer method does not support region-neutral language code identifiers.

Scenario: You are writing an application in which you want the recognizer to be in a specific language, such as Japanese, even if that is not the default recognizer.

Solution: Use the Recognizers.GetDefaultRecognizer method with a language identifier. The sample code that follows shows how to use the default Japanese recognizer, which has an NLS LCID of 0x0411.

C#

Recognizers allRecognizers = new Recognizers();
Recognizer theRecognizer = allRecognizers.GetDefaultRecognizer(0x0411);

Visual Basic .NET

Dim allRecognizers As New Recognizers
Dim theRecognizer As Recognizer = _
    allRecognizers.GetDefaultRecognizer(0x0411)

Common Input Scopes

You can use input scopes to give the recognizer hints about what type of data you expect the user to enter. Input scopes are sometimes referred to as factoids, because linguists use the word factoid to describe a set of constraints that a sequence of characters is expected to meet. Setting the Factoid property on a RecognizerContext object can greatly increase the chance that the recognizer will return results that meet the input scope.

The Tablet PC Platform SDK provides several common input scopes, such as date, Web address, and phone number. The strings for these input scopes can be found on the InputScope Enumeration reference page, in the Tablet PC SDK 1.7.

Scenario: You are building an application in which you expect the user to enter a date in a certain field or location, but the user might also choose to write something else, such as "Unknown."

Solution: Set the RecognizerContext.Factoid property to "IS_DATE_FULLDATE," as shown in the sample code that follows.

C#

theRecognizerContext.Factoid = "IS_DATE_FULLDATE";

Visual Basic .NET

theRecognizerContext.Factoid = "IS_DATE_FULLDATE"

Custom Input Scope

If you are expecting a certain format from the user, but it is not one of the defined input scopes, then you can create your own custom factoid using a subset of .NET regular expressions. To learn about regular expressions to create a custom factoid, read the topics Regular Expression Syntax Reference and Regular Expression Examples in the Tablet PC Platform SDK 1.7.

Scenario: You are building an application in which you expect the user to write down a part number that consists of two letters followed by three numbers.

Solution: Set the RecognizerContext.Factoid property to a string that allows for any letter for the first two characters and any digit for the last three, as shown in the sample code that follows.

C#

string letter = "(a|A|b|B|c|C|d|D|e|E|f|F|g|G|h|H|i|I|j|J|k|K|l|L| " +
    " m|M|n|N|o|O|p|P|q|Q|r|R|s|S|t|T|u|U|v|V|w|W|x|X|y|Y|z|Z)";
string number = "(0|1|2|3|4|5|6|7|8|9)";
theRecognizerContext.Factoid = _
    letter + letter + number + number + number;

Visual Basic .NET

Dim letter As String = _
    "(a|A|b|B|c|C|d|D|e|E|f|F|g|G|h|H|i|I|j|J|k|K|l|L| " + _
    " m|M|n|N|o|O|p|P|q|Q|r|R|s|S|t|T|u|U|v|V|w|W|x|X|y|Y|z|Z)"
Dim number As String = "(0|1|2|3|4|5|6|7|8|9)"
theRecognizerContext.Factoid =
    letter + letter + number + number + number

Input Scope Coercion

Specifying an input scope does not guarantee that the recognition alternates found will be in that format; rather, it only increases the probability that they will. If you always want the recognized text to be appropriate for the input scope, you can set the RecognizerContext.RecognitionFlags property with the RecognitionModes.Coerce flag.

For example, imagine writing the number "13" with the "1" and the "3" extremely close together so that the handwriting resembles a capital letter "B." If you recognize it with no factoid, the recognizer may choose "B" as the first alternate and "13" as another alternate. If you specify the Factoid property as "IS_DIGITS," the recognizer will most likely choose "13" as the first alternate, and "B" as another alternate. If in addition, you set the RecognizerContext.RecognitionFlags with the RecognitionModes.Coerce flag, only numbers will be chosen as alternates.

Note   When you set the RecognizerContext.RecognitionFlags with the RecognitionModes.Coerce flag, the result can also be affected by other recognizer context properties, such as the WordList property.

Scenario: You are building an application that requires the user to enter a date.

Solution: Set the RecognizerContext.Factoid property to "IS_DATE_FULLDATE" and set the RecognizerContext.RecognitionFlags property to coerce the factoid, as shown in the following code sample.

C#

theRecognizerContext.Factoid = "IS_DATE_FULLDATE";
theRecognizerContext.RecognitionFlags |= RecognitionModes.Coerce;

Visual Basic .NET

theRecognizerContext.Factoid = "IS_DATE_FULLDATE"
theRecognizerContext.RecognitionFlags = _
    theRecognizerContext.RecognitionFlags Or RecognitionModes.Coerce;

Word Mode

You can specify that the recognizer look specifically for a single word. In this case, a word is the smallest grouping of strokes that forms a single unit. For recognizers of Latin script, this is usually a word in the recognizer's language, but it can also contain punctuation, numbers, spaces (such as "www.microsoft.com"), and other characters (such as "someone@microsoft.com"). For recognizers of East Asian characters, the single unit is usually a character.

Scenario: You are writing an application in which you expect the user to enter a single word.

Solution: Set the RecognizerContext.RecognitionFlags property with the RecognitionModes.WordMode flag, as shown in the following sample code.

C#

theRecognizerContext.RecognitionFlags |= RecognitionModes.WordMode;

Visual Basic .NET

theRecognizerContext.RecognitionFlags = _
    theRecognizerContext.RecognitionFlags Or RecognitionModes.WordMode;

Word List

If you expect your users to write words that are not in the standard dictionary, then you may want to add those words as a dictionary that applies to your application. For example, your application may need to recognize names of people or industry-specific words, such as the names of chemicals. To add words to a custom dictionary, create a WordList object, add words to it, and then set the RecognizerContext.WordList property. Be aware that if you make any changes to the WordList object, you will need to re-assign it to the RecognizerContext.WordList property in order for those changes to take place.

Note   Recognizers of East Asian characters do not support word lists.
Note   If you set the RecognitionContext.Factoid property to Factoid.WordList and the RecognizerContext.RecognitionFlags property with the RecognitionModes.Coerce flag (the same way that is shown in the preceding section, Input Scope Coercion), then the recognizer will return only words that appear in your WordList object.

Scenario: You are writing an application in which users will be writing industry-specific words that are not in the standard dictionary, and you want the recognizer to be able to recognize them.

Solution: You load in an array of strings that contains each of these words, create a WordList object with these words, and then assign them to the recognizer. The code below shows how to do this for an array of strings named industryWords.

C#

WordList wordList = new WordList();
foreach (string industryWord in industryWords)
{
    wordList.Add(industryWord);
}
theRecognizerContext.WordList = wordList;

Visual Basic .NET

Dim theWordList As New WordList
Dim industryWord As String
For Each industryWord in industryWords
    theWordList.Add(industryWord)
Next
theRecognizerContext.WordList = theWordList

Providing Location and Size Hints

In addition to providing contextual hints, you can also provide information to the recognizer about where the user is likely to write. If you provide a background for the user to write over, you can help guide the user to write letters in expected locations and sizes. Recognition will improve if you provide information to the recognizer such as the number of lines of text, the number of columns in which to write characters, and the height of a midline.

Two common types of backgrounds to write over are lined guides and boxed guides. Lined guides provide one or more lines for the user to write over, such as the writing pad of the Tablet PC Input Panel. Boxed guides provide boxes where the user is expected to write a character in each box, such as the character pad of the Tablet PC Input Panel.

The RecognizerGuide structure provides information about where the user is likely to be writing. You can draw a background for the user to write on in a Paint event. The background will show an area in which to write, as well as a midline. Knowing the position of the midline is especially useful for distinguishing between uppercase and lowercase letters.

A RecognizerGuide specifies a WritingBox property and a DrawnBox property. The writing box collects ink, and the drawn box sets visual cues for the user, who writes within the boundaries. The writing box is larger than the drawn box, providing a margin for the user to stray a little from the drawn box. Specify the boxes and midline in ink space units.

Lined guide

You can provide a lined guide by specifying 0 for either the RecognizerGuide Columns property or the Rows property. A Columns property of zero indicates that the writing is free-form in the horizontal direction, and a Rows property of zero indicates that writing is free-form in the vertical direction. In these situations, you would want to draw a lined background to write in.

Scenario: You've created a special input panel on which the user can write Latin script that has one horizontal row, as shown in the following image. You want to give this information to the recognizer in order to improve recognition.

Solution: Create a Panel object and attach an InkCollector object to it. In the Paint event of the Panel object, draw a box of a different color to show where the user should write (that is, the drawn box), offset by a margin from the borders of the Panel object. Draw a line in the middle of the drawn box. Set the bottom margin to be slightly larger than the other margins so that characters can extend beneath the baseline. Create a RecognizerGuide object and set the Columns property to zero and the Rows property to one. Then perform recognition.

The following sample code shows how to set up the RecognizerGuide object using the RecognizerContext.Guide property. The dimensions of the Panel where the ink is drawn, panelForInk, represent the writing box. The member margin is the margin between the writing box and the drawn box in pixels for the top and the sides of the drawn box. The member bottomMargin is the margin between the bottom of the writing box and the bottom of the drawn box.

C#

// Convert width and height of panel and margins into ink space units.
Renderer renderer = new Renderer();
Graphics panelGraphics = this.panelForInk.CreateGraphics();
Point sizeAsPoint = 
    new Point(this.panelForInk.Width, this.panelForInk.Height);
renderer.PixelToInkSpace(panelGraphics, ref sizeAsPoint);
Point marginsAsPoint = new Point(this.margin, this.bottomMargin);
renderer.PixelToInkSpace(panelGraphics, ref marginsAsPoint);
int marginInkSpace = marginsAsPoint.X;
int bottomMarginInkSpace = marginsAsPoint.Y;
int midlineHeight = 
    (sizeAsPoint.Y - marginInkSpace - bottomMarginInkSpace) / 2;
panelGraphics.Dispose();

// Set recognizer guide to fit the drawn box.
theRecognizerContext.Guide = new RecognizerGuide(
    1, // Number of rows
    0, // Number of columns (freehand)
    midlineHeight, // Height from bottom of drawn box to midline.
    new Rectangle(0, 0, sizeAsPoint.X, sizeAsPoint.Y), // Writing box
        new Rectangle(marginInkSpace, marginInkSpace, // Drawn box
        sizeAsPoint.X - 2 * marginInkSpace, 
        sizeAsPoint.Y - marginInkSpace - bottomMarginInkSpace));

Visual Basic .NET

' Convert width and height of panel and margins into ink space units.
Dim renderer As New Renderer
Dim panelGraphics As Graphics = this.panelForInk.CreateGraphics()
Dim sizeAsPoint As _ 
    New Point(this.PanelForInk.Width, this.PanelForInk.Height)
renderer.PixelToInkSpace(panelGraphics, sizeAsPoint)
Dim marginsAsPoint As New Point(this.margin, this.bottomMargin)
renderer.PixelToInkSpace(panelGraphics, marginsAsPoint)
Dim marginInkSpace As Integer = marginsAsPoint.X
Dim bottomMarginInkSpace As Integer = marginsAsPoint.Y
Dim midlineHeight As Integer = _
    (sizeAsPoint.Y - marginInkSpace - bottomMarginInkSpace) / 2
panelGraphics.Dispose()

' Set recognizer guide to fit the drawn box.
'   1: Number of rows
'   0: Number of columns (freehand)
'   midlineHeight: Height from bottom of drawing area to midline
'   first Rectangle: Writing box
   second Rectangle: Drawn box
guide = New RecognizerGuide(1, 0, midlineHeight, _
    New Rectangle(0, 0, sizeAsPoint.X, sizeAsPoint.Y), _
    New Rectangle(marginInkSpace, marginInkSpace, _
    sizeAsPoint.X - 2 * marginInkSpace, _
    sizeAsPoint.Y - marginInkSpace - bottomMarginInkSpace))

Boxed Guide for Recognizers of East Asian Characters

Recognizers of East Asian characters generally support boxed guides, which makes it easy to have a guide that allows users to write one character in each box. For a boxed guide, you set the Columns and Rows properties to the number of columns and rows of the boxes. When you specify the WritingBox and DrawnBox properties, use the location and size of the box in the top left corner.

Scenario: You've created a special input panel in which the user will write East Asian characters that has one horizontal row with several boxes for each character that you expect, such as is shown in the following diagram. The panel has a background that shows where to write. You want to give this information to the recognizer to improve recognition.

Solution: Create a Panel object and attach an InkCollector object to it. In the Paint event of the Panel object, draw a box for each character, offset by a margin from the borders of the Panel object and from each other. Also, draw a line in the middle of each drawn box. Create a RecognizerGuide object and set the Columns property to the number of boxes and the Rows property to one. Then perform recognition.

The following sample code shows how to set up the RecognizerGuide, where nBoxes is the number of boxes. To see how to set the other variables, look at the preceding Lined Guide example.

C#

theRecognizerContext.Guide = new RecognizerGuide(
    1, // Number of rows
    nBoxes, // Number of columns
    midlineHeight, // Height from bottom of drawing area to midline
    new Rectangle(0, 0, sizeAsPoint.X / nBoxes, sizeAsPoint.Y), // Writing box
    new Rectangle(marginInkSpace, marginInkSpace, // Drawn box
    sizeAsPoint.X / nBoxes - 2 * marginInkSpace, 
    sizeAsPoint.Y - marginInkSpace - bottomMarginInkSpace));

Visual Basic .NET

' Set recognizer guide to fit the drawing area.
'   1: Number of rows
'   nBoxes: Number of columns
'   midlineHeight: Height from bottom of drawing area to midline
'   first Rectangle: Writing box
   second Rectangle: Drawn box
theRecognizerContext.Guide = New RecognizerGuide(
    1, _
    nBoxes, _
    midlineHeight, _
    new Rectangle(0, 0, sizeAsPoint.X / nBoxes, sizeAsPoint.Y), _
    new Rectangle(marginInkSpace, marginInkSpace, _
    sizeAsPoint.X / nBoxes - 2 * marginInkSpace, _
    sizeAsPoint.Y - marginInkSpace - bottomMarginInkSpace))

Boxed Guide for Recognizers of Latin Script

With recognizers of Latin script, you may want to create an input panel with boxes where users write one letter in each box. However, in general, recognizers of Latin script currently do not support boxed guides directly. You can find out if a recognizer supports boxed guides directly by looking for the RecognizerCapabilities.BoxedInput flag in the Recognizer.Capabilities property. You can effectively create a boxed guide by passing in the strokes from each box to the recognizer separately and using the OneChar factoid.

Scenario: You've created a special input panel for entering part numbers that have five Latin characters. You create a panel that contains a background with five boxes. You want to give this information to the recognizer to improve recognition.

Solution: Create a Panel object and attach an InkCollector object to it. In the Paint event of the Panel object, draw one box for each character, offset by a margin from the borders of the Panel object and from each other. Also, draw a line in the middle of each drawn box. The bottom margin should be slightly larger than the rest so that characters can extend beneath the baseline. Create a RecognizerGuide object and set the Columns property to one and the Rows property to one. Loop through each of the boxes and perform recognition with the Factoid property set to OneChar.

The following sample code uses an array of five ComboBox controls, alternateComboBoxes, to display alternates for each of the five boxes in the RecognizerGuide object, using ink from an InkCollector object, theInkCollector. The sample loops through each of the five boxes, extracts the ink from each of them, and then sends the ink to the recognizer. Before this piece of code begins, the RecognizerGuide object is set up as shown in the Boxed Guide for Recognizers of East Asian Characters example. For simplicity, only the C# code follows, but code in Visual Basic .NET is available in the sample projects.

C#

// Set the factoid to look for one character.
theRecognizerContext.Factoid = Factoid.OneChar;

// Loop through each of the five boxes.
for (int i = 0; i < 5; i++)
{
    // Set the recognizer context's strokes using the guide's writing box.
    Ink boxOfInk = 
        theInkCollector.Ink.ExtractStrokes(theRecognizerContext.Guide.WritingBox,
        ExtractFlags.CopyFromOriginal);
        // Check to see if there are any strokes in the box.
        if (boxOfInk.Strokes.Count > 0)
        {
            this.alternateComboBoxes[i].Visible = true;
            theRecognizerContext.Strokes = boxOfInk.Strokes;

            RecognitionStatus status;
            RecognitionResult result = theRecognizerContext.Recognize(out status);
            if (status == RecognitionStatus.NoError)
            {
                // Add alternates to the appropriate combo box.
                this.alternateComboBoxes[i].Items.Clear();
                foreach (RecognitionAlternate alternate in result.GetAlternatesFromSelection())
                {
                    this.alternateComboBoxes[i].Items.Add(alternate);
                }
                this.alternateComboBoxes[i].SelectedIndex = 0;
            }
            else
            {
                MessageBox.Show("Error in recognition:" + status.ToString());
                this.alternateComboBoxes[i].Visible = false;
            }
        }
        else
        {
            // Don't show combo boxes for boxes with no ink.
            this.alternateComboBoxes[i].Visible = false;
        }

    // Move guide over by its width.
    RecognizerGuide oldGuide = this.recognizerContext.Guide;
    RecognizerGuide newGuide = new RecognizerGuide(1, 0, oldGuide.Midline, 
        new Rectangle(oldGuide.WritingBox.Left + oldGuide.WritingBox.Width,
        oldGuide.WritingBox.Top, oldGuide.WritingBox.Width,
        oldGuide.WritingBox.Height), 
        oldGuide.DrawnBox);
    this.recognizerContext.Strokes = null;
    this.recognizerContext.Guide = newGuide;
}

Providing a User Interface to Select Alternates

When recognition occurs, a Recognizer object provides not only its best guess about what the user wrote, but a list of alternative possibilities. The RecognitionAlternates collection contains the possible word matches, and the best guess is found with the TopAlternate property of the RecognitionResult object. You can find more alternates with the RecognitionResult.GetAlternatesFromSelection method. It is up to you to create a user interface that presents the various alternates so that the user can choose the appropriate one. Once the user has selected an alternate word, you can use the RecognitionResult.ModifyTopAlternate method to force the top alternate use the selected alternate. More information about recognition alternates can be found in Using Recognition Alternates.

The following section describes how to display the recognition alternates in a list box. It also describes how to set an option for breaking alternates into separate words. Lastly, it describes how to perform character Autocomplete for East Asian languages.

Show alternates in a control

Once you obtain recognition alternates, it's useful to put them into a control so that users can choose the appropriate alternate word.

Scenario: You are writing a note-taking application in which users can select some of their handwriting and convert it into text. As part of the conversion process, you want to display a ListBox object with the alternate word matches.

Solution: After performing recognition, add each of the alternates into the Items collection of the ListBox object. Because the RecognitionAlternate.ToString method returns the alternate's text, you can add the RecognitionAlternate object to the Items collection directly and it will display correctly in the ListBox object.

The following sample code returns a recognition result from a RecognizerContext object called theRecognizerContext, then uses the strokes from an InkCollector object called theInkCollector, and populates the ListBox object called listBoxAlternates with the alternates.

C#

// Set the recognizer context's strokes.
theRecognizerContext.Strokes = theInkCollector.Ink.Strokes;

// Use the recognizer context to perform recognition on the strokes.
RecognitionStatus status;
RecognitionResult result = theRecognizerContext.Recognize(out status);
if (status == RecognitionStatus.NoError && result != null)
{
    // Display the alternates.
    this.listBoxAlternates.Items.Clear();
    foreach (RecognitionAlternate alternate in 
     result.GetAlternatesFromSelection())
    {
        this.listBoxAlternates.Items.Add(alternate);
    }
}
else
{
    MessageBox.Show("Error in recognition:" + status.ToString());
}

Visual Basic .NET

' Set the recognizer context's strokes.
Me.TheRecognizerContext.Strokes = Me.TheInkCollector.Ink.Strokes

' Use the recognizer context to perform recognition on the strokes.
Dim status As RecognitionStatus
Dim result As RecognitionResult = Me.TheRecognizerContext.Recognize(status)
Me.ListBoxAlternates.Items.Clear()
If Not (result Is Nothing) Then
    If status = RecognitionStatus.NoError AndAlso Not (result Is Nothing) Then
        ' Display the alternates.
        Dim alternate As RecognitionAlternate
        For Each alternate In result.GetAlternatesFromSelection()
            Me.ListBoxAlternates.Items.Add(alternate)
        Next alternate
    Else
        MessageBox.Show("Error in recognition:" + status.ToString())
    End If
End If

Segmentation

When a recognizer analyzes a collection of strokes, it breaks the strokes into groups called segments. For recognizers of Latin script, a segment is usually a word, and for recognizers of East Asian characters, a segment is a character. The number of segments that the recognizer finds can be different for different RecognitionAlternates. For example, if a user prints the word "together," the recognizer may return alternates with one segment ("together"), two segments ("to gather"), or three segments ("to get her").

If you are creating a user interface that allows users to choose alternates for individual segments, you may want to set the RecognizerContext.RecognitionFlags property with the RecognitionModes.TopInkBreaksOnly flag. This forces the recognizer to return only alternates in which the segments match the segments of the top alternate. The Tablet PC Input Panel writing pad is an example of this type of user interface. Note that when you set the TopInkBreaksOnly flag, the recognition speed is faster.

Scenario: You are writing an application with user interface in which ink is recognized and the top alternate is displayed to the user. Under each word is a ComboBox that shows the alternates for that word.

Solution: Set the TopInkBreaksOnly flag for the RecognizerContext object, as shown in the following code sample. Once the alternates are returned, you can populate the combo boxes, ensuring that you do not duplicate a word in a combo box.

C#

theRecognizerContext.RecognitionFlags |=
    RecognitionModes.TopInkBreaksOnly;

Visual Basic .NET

theRecognizerContext.RecognitionFlags = _
    theRecognizerContext.RecognitionFlags Or _
    RecognitionModes.TopInkBreaksOnly;

Character Autocomplete

Most recognizers of East Asian characters can recognize strokes before a user has finished writing the character, which can reduce the number of strokes a user has to make. This is called character Autocomplete. To determine whether a Recognizer object is able to perform character Autocomplete, look for the RecognizerCapabilities.CharacterAutoCompletionInput flag in the Recognizer.Capabilities property. Character Autocomplete works well with a boxed RecognizerGuide object (see the preceding section, Boxed Guide for Recognizers of East Asian Characters).

Character Autocomplete has two modes: Random, which specifies that the order of the strokes can be arbitrary, and Prefix, which specifies that the order of the strokes must conform to the rules of the language. These modes are specified by the RecognizerCharacterAutoCompletionMode enumeration.

Scenario: You are creating an application which recognizes East Asian characters, and you want recognition to occur after each stroke. You would like characters to be recognized as soon as the user has made enough strokes for recognition to occur.

Solution: Set the RecognizerContext.CharacterAutoCompletion property. Add a Stroke event handler to the ink collector, where you call the RecognizerContext.BackgroundRecognize method or the RecognizerContext.BackgroundRecognizeWithAlternates method. Then you'll need to add a Recognition or RecognitionWithAlternates event handler on the RecognizerContext object to display the results when recognition is complete.

The following sample code shows how to set the CharacterAutoCompletion property to RecognizerCharacterAutoCompletionMode.Random, which means that the order of the strokes can be arbitrary for character Autocomplete.

C#

theRecognizerContext.CharacterAutoCompletion = 
    RecognizerCharacterAutoCompletionMode.Random;

Visual Basic .NET

theRecognizerContext.CharacterAutoCompletion = _
    theRecognizerContext.CharacterAutoCompletion Or _
    RecognizerCharacterAutoCompletionMode.Random;

The following sample code shows a Stroke event handler that starts background recognition with alternates.

C#

private void inkCollector_Stroke(object sender, 
Microsoft.Ink.InkCollectorStrokeEventArgs e)
{
    // Stop the unfinished recognition processes.
    theRecognizerContext.StopBackgroundRecognition();

    if (this.recognizerContext.Strokes == null)
    {                   
         // Create an empty strokes collection.
         this.recognizerContext.Strokes =
              this.inkCollector.Ink.CreateStrokes();
    }

    // Add this stroke.
    this.recognizerContext.Strokes.Add(e.Stroke);

Visual Basic .NET

Private Sub inkCollector_Stroke(ByVal sender As Object, _
    ByVal e As Microsoft.Ink.InkCollectorStrokeEventArgs)
    ' If character Autocomplete is enabled, then start background recognition.
    If Me.CheckBoxAutocomplete.Checked Then
        ' Stop the unfinished recognition processes.
        Me.TheRecognizerContext.StopBackgroundRecognition()

        ' Create a new strokes collection if necessary.
        If Me.TheRecognizerContext.Strokes Is Nothing Then
            ' Create an empty strokes collection.
            Me.TheRecognizerContext.Strokes = _
                Me.TheInkCollector.Ink.CreateStrokes()
        End If

        ' Add this stroke.
        Me.TheRecognizerContext.Strokes.Add(e.Stroke)

        ' Start background recognition.
        Me.TheRecognizerContext.BackgroundRecognizeWithAlternates()
    End If
End Sub

Freeform Applications

In the section Providing Location and Size Information, the applications required that the user write in a particular place. However, you may also choose to create a freeform application, in which users can write anywhere within a large area. Microsoft Windows Journal is an example of this kind of application.

This section discusses how to create applications in which the users select strokes to recognize, how to create applications that later analyze the ink for searching or indexing, and how to separate strokes that are intended as text from strokes that are intended as drawings.

User selects strokes to recognize

Scenario: You are developing a note-taking application in which users can select strokes and then convert them into text.

Solution: A common object to use with note-taking applications is the InkOverlay object. When the user initiates the recognition, you need to create a RecognizerContext object and set its Strokes property with the InkOverlay.Selection property. If the user is likely to write words outside the standard dictionary, create a WordList object with those words and set the WordList property of the RecognizerContext object. Call the RecognizerContext.Recognize method and use the RecognitionResult object to display the RecognitionAlternates from which the user can choose an appropriate word match.

Later analysis for search or index

Scenario: You are writing an application that searches through a collection of ink looking for specific text. This could be a user-initiated search through one or more documents, or it could be a batch search for the purposes of creating a searchable index of the ink.

Solution: First you may need to separate the ink that represents drawing from the ink that represents text. For this, you should use the Divider class that is described in the Separating drawings from text section that follows. Create a RecognizerContext object and set its Strokes property (with the results from the Divider object, if you are using one). Create a WordList object if you are expecting words outside the standard dictionary. Call the RecognizerContext.Recognize method and use the RecognitionResult object to obtain all the RecognitionAlternates. For each RecognitionAlternate object that has a high Confidence property, you can match it against your search, or you can put it in your indexing data with a pointer back to the ink.

Note   In freeform applications where the recognition is done in the background, users often write in a messier fashion that is more difficult for the recognizer to work with. This means that you may have to store more recognition alternates in order to find the correct one.

Separating drawings from text

The Divider object analyzes a collection of strokes and divides them into handwriting and pictures. Use the Divider.Divide method to analyze the strokes, and then the DivisionResult.ResultByType method to find lines and paragraphs of text.

Using a Divider object also helps with the recognition of handwriting that is written at an angle. The Divider Sample provides more information about how it is used.

Conclusions

Here is a summary of the information described in this article.

  • You can increase the likelihood that the user gets the correct recognition results by using the features in the Recognizer and RecognizerContext classes.
  • You can select a language using the National Language Support (NLS) Language Code Identifier (LCID).
  • Recognizers of East Asian characters support character Autocomplete and generally do not put in spaces; recognizers of Latin script support word lists.
  • In addition to the recognizer's best guess at handwriting recognition, recognition alternates are available from which the user can choose.
  • You can use input scopes to provide context to the recognizer about the type of content the user might write. With regular expressions, you can specify custom input scopes. You can also require the recognizer to choose only alternates that fit the input scopes, and you can require the recognizer to return only single words.
  • You can create a dictionary of words specific to your application with the WordList object.
  • The RecognizerGuide object provides information about where the user is likely to be writing. You can choose to have lined or boxed guides. Boxed guides are not supported by recognizers of Latin script, but you can achieve the same effect by recognizing the collection of strokes within each box with a OneChar factoid.
  • For some user interfaces, it can be useful to force all alternates to have the same segmentation as the top alternate by setting the TopInkBreaksOnly flag. This results in faster recognition.
  • For East Asian languages, you can often use character Autocomplete, which can reduce the number of strokes a user needs to make per character.
  • You can use the Divider object to separate text from pictures.

Did you find this helpful?
(1500 characters remaining)
Thank you for your feedback
Show:
© 2014 Microsoft. All rights reserved.