Designing the Use of Handwriting Recognition

Design your application based on the uses that the ink data in your application will undergo. If text is not required immediately after a user enters ink, it is best to delay handwriting recognition until the user invokes a tool or command. In some cases, handwriting recognition may never be required. Handwriting recognition in real time, while users apply ink, interrupts them. The user may feel the need to correct a recognition error shortly after it occurs, which takes time and interrupts the primary input task that the user is trying to achieve. Depending on the usage scenario for your application, this may not be the desired behavior.

Leaving ink as ink is valuable in user scenarios in which the user wants to:

• Use free input of handwriting and drawing on a canvas.
• Capture thoughts or take ink notes in a social setting, such as in a meeting or a face-to-face discussion, where notes are primarily meant to be personal.
• Take phone messages.
• Informally and quickly capture action items and reminders.
• Compose personal letters or e-mail messages.
• mark up personal or business documents.
• Fill out forms in fixed size fields, such as digital tax forms.

Using handwriting recognition to convert ink to text is necessary in user scenarios in which the user wants to:

• Input text into edit control fields in dialog boxes that do not support ink, such as saving or opening files or using a filename field.
• Reuse ink as textual data in other applications, such as a user taking ink notes and converting them into a text-based e-mail message.

Tablet PC Input Panel is a dynamic and customizable in-place pen input system for the Tablet PC, dedicated to the specific task of text entry and correction. In Microsoft® Windows® XP Tablet PC Edition with Windows XP Service Pack 2 (SP2), Input Panel improvements include, but are not limited to, discoverability, higher perceived throughput, and reduced user management of the Input Panel window. The goal of Input Panel is to provide easy access to text input with the stylus when and where it is needed by the end user, while minimizing the obstruction of shell and application user interface (UI). The following are guidelines for using handwriting recognition and correction in your application.

• Avoid or reduce the conversion of ink to text. Consider which tasks in your application would benefit from conversion. Your application may benefit from:

  • Making text input easy and intuitive, without becoming obtrusive, by allowing text input when and where it is needed. Improve discoverability through obvious entry points where Input Panel can be activated.

  • Providing unique default names in load or save commands, find dialog boxes, and any other text fields in your application UI in which default values make sense, are likely to be accepted without correction, and reduce the need for user input.

    

  • Using predefined forms with boxes for ink input that have specific context or intent (such as names or addresses). When the context of the ink data is known and the ink can be input only in a fixed area on the screen, recognition is more accurate.

• Do not use handwriting recognition if your application needs only search functionality. Your application can achieve the ability to search ink by keeping some tasks in the background, such as recognition of ink to text and storing the text data (including candidates for recognition) separately, and not showing this to the user. The ink is left as ink on screen. In this manner, the search is performed on the text data, and fuzzy algorithms are applied to compensate for recognition errors. The user is shown which ink words are search results without having to convert the ink to text.
• Include ink as a native data type in your application. Sketches, drawings, and handwritten text are all valuable data that do not require any recognition engines to be called or written.
• Create customized recognizers to allow direct input of your applications native data types that are not text, such as drawing shapes, musical notes, and formulas. Anything that is difficult to draw by using a mouse and keyboard may be much easier to input by using a pen and a stroke-recognition engine that you write.

For more information on writing recognition engines, see About Recognition and Creating a Recognizer.

Note: Handwriting recognition requires correction for conversion errors.

The following are options to consider as correction tools for recognized text:

• Add support for the Microsoft® Windows® Text Services Framework (TSF). TSF provides a simple and scalable framework for integrating text input services—such as Tablet PC Input Panel—into your application. Users then correct recognized text in your application by using Input Panel or an interface that shows recognition candidates for that text. For more information on support that TSF provides, see the Text Services Framework section of MSDN.

  

• Provide a button in your UI so that users can open Input Panel and correct converted text. This is useful in dialogs and forms.

  Note: An application programming interface (API) to launch Input Panel from your application is available. See Using Tablet PC Input Panel for more information.