Cómo convertir trazos de lápiz en texto (HTML)
[ Este artículo está destinado a desarrolladores de Windows 8.x y Windows Phone 8.x que escriben aplicaciones de Windows en tiempo de ejecución. Si estás desarrollando para Windows 10, consulta la documentación más reciente
Aprende a usar el reconocimiento de la escritura a mano y a convertir trazos de lápiz en texto en una aplicación de la Tienda Windows con JavaScript.
Actualizaciones para Windows 8.1: Windows 8.1 incorpora varias actualizaciones y mejoras para las API para entrada de puntero. Para obtener más información, consulta el tema sobre cambios en las API para Windows 8.1.
En este ejemplo, usaremos la funcionalidad de reconocimiento integrada (RecognizeAsync) del objeto InkManager para convertir todos los trazos de lápiz en texto y guardar los resultados en el administrador.
Si no se necesita reconocimiento, usa un objeto InkStrokeContainer en lugar de uno InkManager. Los resultados del reconocimiento anterior siguen disponibles en GetRecognitionResults.
Si solo se requiere el reconocimiento, no el almacenamiento, usa un objeto InkRecognizerContainer en lugar de uno InkManager.
Lo que debes saber
Tecnologías
Requisitos previos
Este tema de procedimientos se basa en Inicio rápido: capturar datos de entrada de lápiz.
Además, damos por hecho que puedes crear una aplicación básica de la Tienda Windows con JavaScript que use la plantilla de Biblioteca de Windows para JavaScript.
- Si quieres obtener instrucciones para crear tu primera aplicación de la Tienda Windows, consulta el tema sobre cómo crear tu primera aplicación de la Tienda Windows con JavaScript.
- Para obtener información sobre cómo usar los objetos y controles de WinJS, consulta Inicio rápido: Agregar controles y estilos de WinJS.
Instrucciones
Paso 1: Crear un administrador de entrada de lápiz
Inicializa un objeto InkManager que procesará y manipulará los datos relacionados con la entrada de lápiz obtenidos de la entrada del puntero.
// Create an ink manager.
// InkManager is documented at https://go.microsoft.com/fwlink/?LinkID=260648.
var inkManager = new Windows.UI.Input.Inking.InkManager();
Paso 2: Definir un idioma predeterminado para el reconocimiento
Define el idioma predeterminado del reconocedor. En Windows 8, cada paquete de idioma incluye un conjunto de idiomas de reconocimiento pertinentes. Para obtener más información sobre los idiomas de reconocimiento válidos, consulta Name.
Si no defines el idioma predeterminado, el sistema seleccionará uno en función del idioma principal que figure en el panel de control Idioma. Si el idioma principal no incluye un idioma de reconocimiento, el sistema empleará uno de los idiomas secundarios o EN-US.
Nota
Te mostramos cómo definir el idioma predeterminado a Reconocedor de escritura a mano para inglés (EE.UU.).
// Set default recognition language.
if (!setRecognizerByName("Microsoft English (US) Handwriting Recognizer")) {
statusMessage.innerText += "\nRecognition: Failed to find English (US) recognizer.";
}
else {
statusMessage.innerText += "\nRecognition: English (US) recognizer.";
}
Usamos GetRecognizers para obtener los idiomas de reconocimiento que hay disponibles en el sistema del usuario. Itera por esta lista y, si encuentras el idioma que quieres, llama a SetDefaultRecognizer. Si no lo encuentras, muestra un mensaje de error y usa el idioma predeterminado del sistema.
/// <summary>
/// Finds a specific recognizer, and sets the inkManager's default to that recognizer.
/// Returns true if successful.
/// </summary>
/// <param name="recognizerName">The name of the handwriting recognizer.</param>
function setRecognizerByName(recognizerName) {
try {
// recognizers is a normal JavaScript array
var recognizers = inkManager.getRecognizers();
for (var i = 0, len = recognizers.length; i < len; i++) {
if (recognizerName === recognizers[i].name) {
inkManager.setDefaultRecognizer(recognizers[i]);
return true;
}
}
}
catch (e) {
displayError("setRecognizerByName: " + e.toString());
}
return false;
}
Paso 3: Habilitar el reconocimiento
Ofrece al usuario una manera de inicia el reconocimiento.
Primero, declara un conjunto de botones de modo.
<body>
<div id="applicationTitle">Ink sample</div>
<div>
<canvas id="inkCanvas"></canvas>
<div>
<button id="load">Load</button>
<button id="save">Save</button>
<button id="draw">Draw</button>
<button id="select">Select</button>
<button id="selectall">Select all</button>
<button id="erase">Erase</button>
<button id="eraseAll">Erase all</button>
<button id="recognize" value="selected">Handwriting recognition</button>
</div>
</div>
<div id="modeMessage"></div>
<div id="deviceMessage"></div>
<div id="statusMessage"></div>
</body>
Después, declara una escucha de eventos.
get("recognize").addEventListener("click", recognizeStrokes, false);
Paso 4: Procesar todos los trazos
Por último, administra la solicitud de reconocimiento y procesa todos los trazos administrados por el administrador de entrada de lápiz mediante el motor de reconocimiento.
Para que sea más sencillo, en este ejemplo todos los trazos se procesan especificando un InkRecognitionTarget de all. Al especificar un InkRecognitionTarget de recent, solo se procesan aquellos trazos que se agregaron desde la última llamada a RecognizeAsync. Esto suele ser útil cuando todos los trazos se tratan como escritura a mano y deben reconocerse en forma dinámica en función de un evento como pointerup, en lugar de un comando de usuario. Usa selected para procesar únicamente los trazos seleccionados.
/// <summary>
/// Invoked when the "Handwriting recognition" button is pressed.
/// The ink manager processes all strokes through recognizeAsync, which returns
/// the number of words detected and a set of recognition results for each word.
/// </summary>
/// <param name="evt">The event object.</param>
function recognizeStrokes(evt) {
// Ensure ink strokes exist before calling recognizeAsync.
if (inkManager.getStrokes().length > 0) {
// recognizeAsync fails if other recognition tasks are in progress.
try {
// The ink manager is used to store the recognition results.
// recognizeAsync has 3 modes: all | selected | recent.
// For this example, we process all strokes.
// recognizeAsync is documented at https://go.microsoft.com/fwlink/?LinkID=265172.
inkManager.recognizeAsync(Windows.UI.Input.Inking.InkRecognitionTarget.all).done
(
// The recognitionResult object returned by recognizeAsync exposes the
// bounding rect, strokes, and text candidates for each word.
// In this example, we simply display the word count and recognition results.
function (results) {
// recognizeAsync does not automatically update existing recognition results in the ink manager.
// updateRecognitionResults is documented at https://go.microsoft.com/fwlink/?LinkID=265175.
inkManager.updateRecognitionResults(results);
var x = inkManager.getRecognizers();
// Display the number of words returned in results.
statusMessage.innerText = "Words recognized: " + results.length.toString();
// Iterate through each word and display the ranked list of possible matches.
for (var i = 0; i < results.length; i++) {
statusMessage.innerText += "\nWord" + (i+1).toString() + ":";
var alts = results[i].getTextCandidates();
for (var j = 0; j < alts.length; j++) {
statusMessage.innerText += " " + alts[j].toString();
}
}
},
function (e) {
displayError("InkManager::recognizeAsync: " + e.toString());
}
);
}
catch (e) {
displayError("recognize: " + e.toString());
}
}
else {
statusMessage.innerText = "No strokes to recognize.";
}
}
Consulta Entrada: muestra de entrada de lápiz para ver un ejemplo de cómo usar el rectángulo delimitador y los candidatos de texto para cada palabra que se expone en la colección de objetos InkRecognitionResult que RecognizeAsync devuelve. Una vez que finalizó el reconocimiento, pulsa una palabra para mostrar un control flotante al lado de la palabra con una lista de los candidatos de texto que puedes seleccionar.
Ejemplo completo
//// THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF
//// ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO
//// THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A
//// PARTICULAR PURPOSE.
////
//// Copyright (c) Microsoft Corporation. All rights reserved
// Windows Store app that demonstrates the use of the Windows.UI.Input.Inking APIs.
// Ink functionality is documented at https://go.microsoft.com/fwlink/?LinkID=260649.
// User interaction functionality is documented at https://go.microsoft.com/fwlink/?LinkID=260650.
// Ink APIs are documented at https://go.microsoft.com/fwlink/?LinkID=260652.
// Pointer APIs are documented at https://go.microsoft.com/fwlink/?LinkID=260653.
(function ()
{
"use strict";
//
// Global variables
//
// UI object references.
var inkCanvas;
var inkContext;
var modeMessage;
var deviceMessage
var statusMessage;
// Create an ink manager.
// InkManager is documented at https://go.microsoft.com/fwlink/?LinkID=260648.
var inkManager = new Windows.UI.Input.Inking.InkManager();
// Initial pointer values.
var pointerId = -1;
var pointerDeviceType = null;
// Initial stroke property values.
var strokeColor;
var strokeWidth;
//
// End global variables
//
// Obtain reference to the specified element.
function get(elementId)
{
return document.getElementById(elementId);
}
function initialize()
{
// Set up the UI.
inkCanvas = get("inkCanvas");
inkContext = inkCanvas.getContext("2d");
inkContext.lineCap = "round";
inkContext.lineKJoin = "round";
inkCanvas.width = window.innerWidth - 10;
inkCanvas.height = window.innerHeight * 0.5;
deviceMessage = get("deviceMessage");
deviceMessage.innerText = "Undefined";
modeMessage = get("modeMessage");
modeMessage.innerText = inkManager.mode;
statusMessage = get("statusMessage");
statusMessage.innerText = "No pointer input detected."
// Set initial ink mode.
drawStrokes();
// Set default recognition language.
if (!setRecognizerByName("Microsoft English (US) Handwriting Recognizer")) {
statusMessage.innerText += "\nRecognition: Failed to find English (US) recognizer.";
}
else {
statusMessage.innerText += "\nRecognition: English (US) recognizer.";
}
// Set up the handlers for input processing.
inkCanvas.addEventListener("pointerdown", onPointerDown, false);
inkCanvas.addEventListener("pointermove", onPointerMove, false);
inkCanvas.addEventListener("pointerup", onPointerUp, false);
get("save").addEventListener("click", saveStrokes, false);
get("load").addEventListener("click", loadStrokes, false);
get("draw").addEventListener("click", drawStrokes, false);
get("select").addEventListener("click", selectStrokes, false);
get("selectall").addEventListener("click", selectAllStrokes, false);
get("erase").addEventListener("click", eraseStrokes, false);
get("eraseAll").addEventListener("click", eraseAllStrokes, false);
get("recognize").addEventListener("click", recognizeStrokes, false);
}
document.addEventListener("DOMContentLoaded", initialize, false);
function getPointerDeviceType(pId)
{
var pointerDeviceType;
var pointerPoint = Windows.UI.Input.PointerPoint.getCurrentPoint(pId);
switch (pointerPoint.pointerDevice.pointerDeviceType)
{
case Windows.Devices.Input.PointerDeviceType.touch:
pointerDeviceType = "Touch";
break;
case Windows.Devices.Input.PointerDeviceType.pen:
pointerDeviceType = "Pen";
break;
case Windows.Devices.Input.PointerDeviceType.mouse:
pointerDeviceType = "Mouse";
break;
default:
pointerDeviceType = "Undefined";
}
deviceMessage.innerText = pointerDeviceType;
return pointerDeviceType;
}
// Occurs when the pointer (touch, pen, mouse) is detected by the canvas.
// Each stroke begins with onPointerDown.
function onPointerDown(evt)
{
// Get the device type for the pointer input.
pointerDeviceType = getPointerDeviceType(evt.pointerId);
// Process pen and mouse (with left button) only. Reserve touch for manipulations.
if ((pointerDeviceType === "Pen") || ((pointerDeviceType === "Mouse") && (evt.button === 0)))
{
statusMessage.innerText = pointerDeviceType + " pointer down: Start stroke. "
// Process one pointer at a time.
if (pointerId === -1)
{
var current = evt.currentPoint;
// Start drawing the stroke.
inkContext.beginPath();
inkContext.lineWidth = strokeWidth;
inkContext.strokeStyle = strokeColor;
inkContext.moveTo(current.position.x, current.position.y);
// Add current pointer to the ink manager (begin stroke).
inkManager.processPointerDown(current);
// The pointer id is used to restrict input processing to the current stroke.
pointerId = evt.pointerId;
}
}
else
{
// Process touch input.
}
}
// Mouse: Occurs when the pointer moves.
// Pen/Touch: Occurs at a steady rate (approx. 100 messages/second) whether the pointer moves or not.
function onPointerMove(evt)
{
// Process pen and mouse (with left button) only. Reserve touch for manipulations.
if ((pointerDeviceType === "Pen") || ((pointerDeviceType === "Mouse") && (evt.button === -1)))
{
statusMessage.innerText = pointerDeviceType + " pointer move: Draw stroke as lines. "
// The pointer Id is used to restrict input processing to the current stroke.
// pointerId is updated in onPointerDown().
if (evt.pointerId === pointerId)
{
var current = evt.currentPoint;
// Draw stroke in real time.
inkContext.lineTo(current.rawPosition.x, current.rawPosition.y);
inkContext.stroke();
// Add current pointer to the ink manager (update stroke).
inkManager.processPointerUpdate(current);
}
}
else
{
// Process touch input.
}
}
// Occurs when the pointer (touch, pen, mouse) is lifted from the canvas.
// Each stroke ends with onPointerUp.
function onPointerUp(evt)
{
// Process pen and mouse (with left button) only. Reserve touch for manipulations.
if ((pointerDeviceType === "Pen") || ((pointerDeviceType === "Mouse") && (evt.button === 0)))
{
statusMessage.innerText = pointerDeviceType + " pointer up: Finish stroke. "
if (evt.pointerId === pointerId) {
// Add current pointer to the ink manager (end stroke).
inkManager.processPointerUp(evt.currentPoint);
// End live drawing.
inkContext.closePath();
// Render strokes using bezier curves.
renderAllStrokes();
// Reset pointer Id.
pointerId = -1;
}
}
else
{
// Process touch input.
}
}
// Render all strokes using bezier curves instead of line segments.
function renderAllStrokes()
{
statusMessage.innerText += "Render strokes as bezier curves."
// Clear the drawing surface of existing strokes.
inkContext.clearRect(0, 0, inkCanvas.width, inkCanvas.height);
// Iterate through each stroke.
inkManager.getStrokes().forEach(
function (stroke)
{
inkContext.beginPath();
if (stroke.selected) {
inkContext.lineWidth = stroke.drawingAttributes.size.width * 2;
inkContext.strokeStyle = "green";
} else {
inkContext.lineWidth = stroke.drawingAttributes.size.width;
inkContext.strokeStyle = "black";
}
// Enumerate through each line segment of the stroke.
var first = true;
stroke.getRenderingSegments().forEach(
function (segment)
{
// Move to the starting screen location of the stroke.
if (first)
{
inkContext.moveTo(segment.position.x, segment.position.y);
first = false;
}
// Calculate the bezier curve for the segment.
else
{
inkContext.bezierCurveTo(segment.bezierControlPoint1.x,
segment.bezierControlPoint1.y,
segment.bezierControlPoint2.x,
segment.bezierControlPoint2.y,
segment.position.x, segment.position.y);
}
}
);
// Draw the stroke.
inkContext.stroke();
inkContext.closePath();
}
);
}
// Set up draw mode.
function drawStrokes() {
inkManager.mode = Windows.UI.Input.Inking.InkManipulationMode.inking;
strokeColor = "black";
strokeWidth = 2;
modeMessage.innerText = inkManager.mode;
}
// Set up selection mode.
function selectStrokes() {
inkManager.mode = Windows.UI.Input.Inking.InkManipulationMode.selecting;
strokeColor = "red";
strokeWidth = 1;
modeMessage.innerText = inkManager.mode;
}
// Set up erase mode.
function eraseStrokes() {
inkManager.mode = Windows.UI.Input.Inking.InkManipulationMode.erasing;
strokeColor = "gold";
strokeWidth = 1;
modeMessage.innerText = inkManager.mode;
}
// Select all strokes handler.
function selectAllStrokes() {
inkManager.mode = Windows.UI.Input.Inking.InkManipulationMode.selecting;
strokeColor = "red";
strokeWidth = 1;
modeMessage.innerText = "Select all strokes.";
// Iterate through each stroke.
inkManager.getStrokes().forEach(
function (stroke) {
stroke.selected = 1;
}
);
renderAllStrokes();
}
// Select all strokes handler.
function eraseAllStrokes() {
inkManager.mode = Windows.UI.Input.Inking.InkManipulationMode.inking;
strokeColor = "black";
strokeWidth = 2;
modeMessage.innerText = "Erase all strokes.";
// Iterate through each stroke.
inkManager.getStrokes().forEach(
function (stroke) {
stroke.selected = 1;
}
);
inkManager.deleteSelected();
renderAllStrokes();
}
// Save all strokes owned by inkManager.
function saveStrokes()
{
// Ensure that strokes exist before calling saveAsync.
if (inkManager.getStrokes().size > 0)
{
// Set up the file save screen.
var savePicker = Windows.Storage.Pickers.FileSavePicker();
savePicker.suggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.picturesLibrary;
savePicker.fileTypeChoices.insert("GIF with embedded ISF", [".gif"]);
savePicker.defaultFileExtension = ".gif";
// Set up the stream.
var saveStream = null;
// Asynchronously save the ink data to the stream.
savePicker.pickSaveFileAsync().done(
function (file)
{
if (null !== file)
{
file.openAsync(Windows.Storage.FileAccessMode.readWrite).then(
function (stream)
{
saveStream = stream;
return inkManager.saveAsync(saveStream);
}
).then(
function ()
{
return saveStream.flushAsync();
},
function (e) {
// Override the standard saveAsync error with our own.
throw new Error("saveAsync");
}
).done(
function ()
{
statusMessage.innerText = "Strokes saved as GIF with embedded ISF (.gif).";
saveStream.close();
},
function (e) {
statusMessage.innerText = "Save: " + e.toString();
// Close the stream if open.
if (saveStream) {
saveStream.close();
}
}
);
}
}
);
}
else
{
statusMessage.innerText = "No strokes to save.";
}
}
// Load strokes into an inkManager.
function loadStrokes()
{
// Set up the file open screen.
var openPicker = Windows.Storage.Pickers.FileOpenPicker();
openPicker.suggestedStartLocation = Windows.Storage.Pickers.PickerLocationId.picturesLibrary;
openPicker.fileTypeFilter.replaceAll([".gif"]);
// Set up the stream.
var loadStream = null;
// Asynchronously load the ink data from the stream.
openPicker.pickSingleFileAsync().done(
function (file)
{
if (null != file)
{
file.openAsync(Windows.Storage.FileAccessMode.read).then(
function (stream) {
loadStream = stream;
return inkManager.loadAsync(loadStream);
}).done(
function()
{
var strokes = inkManager.getStrokes().length;
if (strokes === 0)
{
statusMessage.innerText = "No strokes in file.";
}
else
{
statusMessage.innerText = strokes + " strokes loaded.";
}
renderAllStrokes();
loadStream.close();
},
function (e)
{
statusMessage.innerText = "Load failed.";
if (loadStream)
{
// Close the stream if open.
loadStream.close();
}
});
}
});
}
/// <summary>
/// Finds a specific recognizer, and sets the inkManager's default to that recognizer.
/// Returns true if successful.
/// </summary>
/// <param name="recognizerName">The name of the handwriting recognizer.</param>
function setRecognizerByName(recognizerName) {
try {
// recognizers is a normal JavaScript array
var recognizers = inkManager.getRecognizers();
for (var i = 0, len = recognizers.length; i < len; i++) {
if (recognizerName === recognizers[i].name) {
inkManager.setDefaultRecognizer(recognizers[i]);
return true;
}
}
}
catch (e) {
displayError("setRecognizerByName: " + e.toString());
}
return false;
}
/// <summary>
/// Invoked when the "Handwriting recognition" button is pressed.
/// The ink manager processes all strokes through recognizeAsync, which returns
/// the number of words detected and a set of recognition results for each word.
/// </summary>
/// <param name="evt">The event object.</param>
function recognizeStrokes(evt) {
// Ensure ink strokes exist before calling recognizeAsync.
if (inkManager.getStrokes().length > 0) {
// recognizeAsync fails if other recognition tasks are in progress.
try {
// The ink manager is used to store the recognition results.
// recognizeAsync has 3 modes: all | selected | recent.
// For this example, we process all strokes.
// recognizeAsync is documented at https://go.microsoft.com/fwlink/?LinkID=265172.
inkManager.recognizeAsync(Windows.UI.Input.Inking.InkRecognitionTarget.all).done
(
// The recognitionResult object returned by recognizeAsync exposes the
// bounding rect, strokes, and text candidates for each word.
// In this example, we simply display the word count and recognition results.
function (results) {
// recognizeAsync does not automatically update existing recognition results in the ink manager.
// updateRecognitionResults is documented at https://go.microsoft.com/fwlink/?LinkID=265175.
inkManager.updateRecognitionResults(results);
var x = inkManager.getRecognizers();
// Display the number of words returned in results.
statusMessage.innerText = "Words recognized: " + results.length.toString();
// Iterate through each word and display the ranked list of possible matches.
for (var i = 0; i < results.length; i++) {
statusMessage.innerText += "\nWord" + (i+1).toString() + ":";
var alts = results[i].getTextCandidates();
for (var j = 0; j < alts.length; j++) {
statusMessage.innerText += " " + alts[j].toString();
}
}
},
function (e) {
displayError("InkManager::recognizeAsync: " + e.toString());
}
);
}
catch (e) {
displayError("recognize: " + e.toString());
}
}
else {
statusMessage.innerText = "No strokes to recognize.";
}
}
// Returns true if any strokes inside the ink manager are selected; false otherwise.
function anySelected() {
var strokes = inkManager.getStrokes();
var len = strokes.length;
for (var i = 0; i < len; i++) {
if (strokes[i].selected) {
return true;
}
}
return false;
}
})();
Temas relacionados
Conceptos
Responder a la entrada de pluma y lápiz
Inicio rápido: Capturar datos de lápiz
Referencia
Muestras (DOM)
Entrada: muestra de control de eventos de puntero de DOM
Muestras (API de aplicaciones de la Tienda Windows)
Entrada: muestra de funcionalidades del dispositivo