Export (0) Print
Expand All
Add Support for Digital Ink to Your Windows Applications
Power to the Pen: The Pen is Mightier with GDI+ and the Tablet PC Real-Time Stylus
Expand Minimize

Creating a Scalable Ink Picture Control for Tablet PC

 

Leszynski Group, Inc.

March 2006

Applies to:
   Microsoft Tablet PC Platform SDK
   Microsoft Windows XP Tablet PC Edition 2005
   Microsoft Visual Studio .NET 2003

Summary: This article and sample code demonstrate how to write your own custom control to collect scalable ink annotations on graphics, with rich editing abilities, on a Tablet PC. (35 printed pages)

Click here to download the code sample for this article.

Contents

Introduction
ScalableInkPicture Overview
ScalableInkPicture Architecture
   Editing Modes
   Cut, Copy, Paste, Undo, and Redo
   Scaling and Zooming
   Panning
   Printing
   Serialization
ScalableInkPicture on the Web
Extending ScalableInkPicture
Conclusion
Biography

Introduction

This article and sample code provide an example of how to write a robust, custom annotation control for use in Windows and Web applications. The control mimics the structure of InkPicture, but adds support for synchronized zooming, panning, and printing of ink annotations and an underlying image. The control includes in-place and pop-up editing modes. The sample code creates a component called ScalableInkPicture that demonstrates how to:

  • Write a single control for use on Windows Forms and Web Forms.
  • Support full-featured, in-place and pop-up form editing.
  • Implement cut, copy, paste, undo, and redo for ink operations.
  • Scale ink strokes, pen tip, and brush size when zooming in and out, in synchronization with an underlying image.
  • Pan an enlarged and annotated image, including how to handle scrollbars.
  • Serialize ink for persistence and communication.
  • Print graphics with ink annotations, scaling the ink and image as needed.
  • Address Web security issues related to ink.

The custom control is contained in an assembly named Microsoft.Samples.TabletPC.ScalableInkPicture.dll that you can use in your applications as-is or with customization.

The sample source code was created in Visual Studio 2003 (.NET Framework 1.1) using C#. The sample also includes compiled binaries that can be demonstrated without Visual Studio. For instructions about installing, compiling, and demonstrating the sample, see Using the ScalableInkPicture Sample (UsingScalableInkPicture.doc) in the root folder of the sample solution.

The sample code can be converted successfully to Visual Studio 2005 without errors but has not been extensively tested with this version.

Because there are many different sizes, shapes, and file types available in a corporate image library, additional error handling and testing of ScalableInkPicture is recommended before using this control in production scenarios.

ScalableInkPicture Overview

ScalableInkPicture provides a simple, yet realistic, example of the type of user interface controls that can be built with the Tablet PC SDK to improve or customize the user experience in software solutions. In many types of solutions, you will want to enable users to collect ink annotations on photos, scanned documents, clip art, or other graphical images. The InkPicture control in the Tablet PC SDK provides a great starting point for such use cases, but it does not inherently expose certain capabilities that are vital for full-featured applications, for example, rich in-place editing, zooming, scaling, panning, or scrolling.

ScalableInkPicture is a superset of InkPicture's capabilities, enabling the user to scale the image by zooming in or out. When zoomed, ScalableInkPicture relocates and resizes the ink annotations to match the proportions of the image. ScalableInkPicture also proportionally resizes the pen tip and highlighter brush.

The control has two modes. In In-place mode, ScalableInkPicture is placed on a form in the same manner that you would locate a Windows Forms PictureBox control, sized to fit within the boundaries of the form. In this mode, the control supports in-place annotations on the underlying image. In-place mode also exposes a context menu to choose an annotation tool (pen, highlighter, and eraser) and to use the Clipboard and printer. Figure 1 shows annotations made by using ScalableInkPicture in In-place mode.

Figure 1. In-place annotations made by using ScalableInkPicture in In-Place mode.

In Pop-up mode, ScalableInkPicture is placed on a form with the image serving as a thumbnail or hotspot. In this mode, the image is usually sized fairly small in order to provide space on the form for other controls. When you tap ScalableInkPicture in Pop-up mode, it launches a full-screen form with an annotation toolbar and additional features that including zooming, scaling, panning, scrolling, and printing.

Figure 2 shows the image from Figure 1 in Pop-up mode, zoomed-in to 250%. The previously created ink is scaled to reflect the zoom factor. The current pen tip is also scaled so that the new annotations are proportionally sized.

Figure 2. Annotations made by using the ScalableInkPicture in Pop-up mode.

Figure 3 shows annotations made using ScalableInkPicture on a Web page. As contrasted with the toolbar in Figure 2, the control's toolbar in Figure 3 does not include the Print and Clipboard buttons. When ScalableInkPicture is used on the Web, these features are disabled for security reasons.

Figure 3. ScalableInkPicture supports in-place and pop-up editing on a Web page

ScalableInkPicture Architecture

The ScalableInkPicture component includes the following classes:

  • ScalableInkPictureControl. The top-level object, sub-classing a custom WebInkPicture control, providing in-place editing features and pop-up form functionality through an embedded ScalableInkPictureForm object.
  • WebInkPicture. A simple, Web control, designed with matters os security and trust in mind. WebInkPicture derives from PictureBox, with an InkOverlay added for ink annotations. This base control intentionally uses the same interface (properties, methods, and events) as InkPicture. This approach enables you to easily modify ScalableInkPicture to derive from InkPicture instead, if your needs do not include Web-compliance or otherwise would benefit from InkPicture.
  • ScalableInkPictureForm. A pop-up form that is invoked by the user at run-time by tapping the ScalableInkPictureControl or by clicking Edit on the right-click context menu. This pop-up form provides the zooming and panning abilities of the control.
  • DibGraphicsBuffer. Used for printing ink and for merging ink into an image. For more information about this process, see Printing Ink.

ScalableInkPicture derives from the WebInkPicture base class, which in turn derives from PictureBox with an InkOverlay control added. As described in ScalableInkPicture on the Web, later in this article, this architecture was chosen for Web security reasons. If use in a Web browser was not a functional requirement of ScalableInkPicture, a legitimate alternate architecture for the component would be to derive from the InkPicture control instead.

Editing Modes

ScalableInkPicture provides both in-place editing and pop-up form editing in order to support a full-featured user experience. By default, ScalableInkPictureControl is in Pop-up mode and invokes its ScalableInkPictureForm when the user taps the control. To avoid creating a stroke or a dot when this occurs, the RasterOperation is set to NoOperation by default:

DefaultDrawingAttributes.RasterOperation = RasterOperation.NoOperation;

In-place mode is enabled by setting the InEditMode property to true. In this mode, the context menu is enabled and the pen is configured:

/// <summary>
/// Gets or sets whether the control is in in-place edit mode 
/// (as opposed to tap-to-edit mode).
/// </summary>
[DefaultValue(false), Category("Ink Control"), 
   Description("Indicates if the control is editing in-place or is in tap-to-zoom mode.")]
public bool InEditMode
{
   get
   {
      return inEditMode;
   }
   set
   {
      inEditMode = value;
      ContextMenu = null;

      if (inEditMode)
      {
         initialInkString = InkString;
         if ((enableContextMenu) && (Image != null))
         {
            ContextMenu = contextMenu;
         }
         if (Image != null)
         {
            // Restore default pen color.
            Pen(Color.Blue);
         }
      }
      else
      {
         DefaultDrawingAttributes.RasterOperation = RasterOperation.NoOperation;
      }
      SetCursor();
      InkEnabled = (Image != null);
   }
}

The InEditMode code sets the initialInkString to InkString, which is a serialized version of the InkOverlay's current ink. This provides a mechanism to check if the user has made changes while in edit mode:

/// <summary>
/// Gets a value that indicates whether the ink has changed since the last time it was saved.
/// </summary>
[DefaultValue(""), Category("Ink Control"), Description
   ("Indicates whether ink has changed and the control has unsaved edits.")]
public bool InkChanged
{
   get
   {
      return (initialInkString != InkString);
   }
}

The InEditMode property setting invokes the Pen method, which sets the RasterOperation to CopyPen so that pen strokes are captured by the InkOverlay:

//
// Pen
//
// Sets the current editing tool to the Pen.
//
// Parameters:
//  color - The desired color for the Pen.
//
private void Pen(Color color)
{
   bool editToolChanged = (editTool != EditTool.Pen);
   bool penColorChanged = (color != penColor);
   editTool = EditTool.Pen;
   penColor = color;         
   DrawingAttributes drawingAttributes = new DrawingAttributes();
   drawingAttributes.Color = penColor;
   drawingAttributes.RasterOperation = RasterOperation.CopyPen;
   DefaultDrawingAttributes = drawingAttributes;
   SetEditingMode(InkOverlayEditingMode.Ink);
   if (editToolChanged)
   {
      OnEditToolChanged(new EventArgs());
   }
   if (penColorChanged)
   {
      OnPenColorChanged(new EventArgs());
   }
   SetContextMenuCheck();
}

To avoid a stroke or a dot on the InkOverlay when the user right-clicks the control, set the RasterOperation to NoOperation when the right-click is processed. After, set it back to its prior value:

/// <summary>
/// Raises the CursorButtonDown event.
/// </summary>
/// <param name="e">An InkCollectorCursorButtonDownEventArgs that contains the event data.</param>
/// <remarks>
/// This method is overridden to keep ink from being created when the user simulates a 
/// right-click by pressing the barrel button on the pen.
/// </remarks>
protected override void OnCursorButtonDown(InkCollectorCursorButtonDownEventArgs e)
{
   foreach (CursorButton button in e.Cursor.Buttons)
   {
      if (button.Name == "Barrel Switch")
      {
         if ((button.State == CursorButtonState.Down)
            && !rightClicking)
         {
            preRightClickRasterOp = DefaultDrawingAttributes.RasterOperation;
            DefaultDrawingAttributes.RasterOperation = RasterOperation.NoOperation;
            rightClicking = true;
         }
         break;
      }
   }

   base.OnCursorButtonDown (e);
}

/// <summary>
/// Raises the Stroke event.
/// </summary>
/// <param name="e">An InkCollectorStrokeEventArgs that contains the event data.</param>
protected override void OnStroke(InkCollectorStrokeEventArgs e)
{
   base.OnStroke (e);

   if (rightClicking)
   {
      rightClicking = false;
      DefaultDrawingAttributes.RasterOperation = preRightClickRasterOp;
      return;
   }
   
   if (editTool == EditTool.Lasso)
   {
      return;
   }
   
   if (Cursor != System.Windows.Forms.Cursors.Default)
   {
      // The hand pointer or other non-default cursor.
      Ink.DeleteStroke(e.Stroke);
      return;
   }

   if (erasing)
   {
      erasing = false;
      return;
   }

   SaveUndoState();

   // Remove this current stroke from the undo state.
   previousInk.DeleteStroke(previousInk.Strokes[previousInk.Strokes.Count - 1]);
}

In-place editing exposes a context menu to control pen and highlighter colors and to provide other editing functions. The context menu includes an Edit command to provide access to the ScalableInkPictureForm (the Pop-up mode) even when In-place mode is enabled. Figure 4 shows the context menu.

Figure 4. ScalableInkPicture context menu.

Cut, Copy, Paste, Undo, and Redo

The Cut, Copy, Paste, Undo, and Redo commands are accessed through the In-place mode's context menu. These commands are also on the toolbar in Pop-up mode, and are available programmatically.

The Cut method moves the selection to the Clipboard:

/// <summary>
/// Cuts the selection to the Clipboard.
/// </summary>
public void Cut()
{
   if (!clipboardEnabled || (Selection.Count == 0))
   {
      return;
   }
   
   SaveUndoState();

   // Copy, then delete, the selected strokes.
   Copy();
   ClearSelection();
   Refresh();
}

The Copy method copies the selection to the Clipboard:

/// <summary>
/// Copies the selection to the Clipboard.
/// </summary>
public void Copy()
{
   if (!clipboardEnabled || (Selection.Count == 0))
   {
      return;
   }

   // Copy to the Clipboard.
   Ink.ClipboardCopy(Selection, InkClipboardFormats.Default, InkClipboardModes.Copy);
}

The Paste method retrieves strokes from the Clipboard:

/// <summary>
/// Pastes the Clipboard data.
/// </summary>
/// <exception cref="System.Runtime.InteropServices.COMException">The paste failed.</exception>
public void Paste()
{
   if (clipboardEnabled && Ink.CanPaste())
   {
      SaveUndoState();

      // Compute the location where the ink should be pasted;
      // this location is shifted from the original
      // location so that if the user performs a copy followed 
      // immediately by a paste, the pasted ink won't be directly
      // on top of the copied ink.
      Point location = new Point(13, 13);
      using (Graphics g = CreateGraphics())
      {
         Renderer.PixelToInkSpace(g, ref location);
      }

      // If there is something currently selected, offset it from that.
      if (Selection.Count > 0)
      {
         Point p = Selection.GetBoundingBox().Location;
         location.Offset(p.X, p.Y);
      }

      try
      {
         // Paste the Clipboard data into the Ink.
         Strokes pastedStrokes = Ink.ClipboardPaste(location);

         // Select the pasted strokes.
         Selection = pastedStrokes;

         // Switch to Lasso to show the selected strokes.
         EditTool = EditTool.Lasso;
      }
      catch (System.Runtime.InteropServices.COMException cex)
      {
         System.Diagnostics.Debug.WriteLine("ScalableInkPictureForm::Paste exception: " + cex.Message);

         if ((uint)cex.ErrorCode == 0x80040064)
         {
            // Error is 'Invalid FORMATETC structure', usually caused by failure while pasting ink.
         }
         else
         {
            throw cex;
         }
      }
   }
}

The Undo method undoes the most recent action, such as drawing or erasing a stroke. Undo is only one level deep:

/// <summary>
/// Preserves the current ink state to return to when an Undo command is received.
/// </summary>
public void SaveUndoState()
{
   bool oldCanUndo = CanUndo;
   bool oldCanRedo = CanRedo;
   if (previousInk != null)
   {
      previousInk.Dispose();
   }
   previousInk = Ink.Clone();
   if (nextInk != null)
   {
      nextInk.Dispose();
      nextInk = null;
   }
   if (CanUndo != oldCanUndo)
   {
      OnCanUndoChanged(new EventArgs());
   }
   if (CanRedo != oldCanRedo)
   {
      OnCanRedoChanged(new EventArgs());
   }
}

/// <summary>
/// Undoes the latest edit.
/// </summary>
public void Undo()
{
   if (previousInk != null)
   {
      nextInk = Ink.Clone();
      // Ink must be disabled while it's being replaced.
      InkEnabled = false;
      Ink = previousInk.Clone();
      InkEnabled = true;
      previousInk.Dispose();
      previousInk = null;
      Refresh();
      OnCanUndoChanged(new EventArgs());
      OnCanRedoChanged(new EventArgs());
   }
}

The single action in the undo stack is set during various activities. For example, resizing a selection of strokes calls SaveUndoState:

/// <summary>
/// Raises the SelectionResizing event.
/// </summary>
/// <param name="e">An InkOverlaySelectionResizingEventArgs that contains the event data.</param>
protected override void OnSelectionResizing(InkOverlaySelectionResizingEventArgs e)
{
   base.OnSelectionResizing (e);

   SaveUndoState();
}

The Redo method uses the nextInk value created by the Undo method to restore altered strokes:

/// <summary>
/// Redoes the latest edit.
/// </summary>
public void Redo()
{
   if (nextInk != null)
   {
      previousInk = Ink.Clone();
      // Ink must be disabled while it's being replaced.
      InkEnabled = false;
      Ink = nextInk.Clone();
      InkEnabled = true;
      nextInk.Dispose();
      nextInk = null;
      Refresh();
      OnCanUndoChanged(new EventArgs());
      OnCanRedoChanged(new EventArgs());
   }
}

The CanUndoChanged and CanRedoChanged events are raised by undo and redo to alert the control to enable or disable the Undo and Redo commands as appropriate.

Scaling and Zooming

The user zooms in the ScalableInkPicture pop-up by using the Zoom combo box and the Zoom In and Zoom Out toolbar buttons. The Zoom combo box provides a list of seven fixed zoom percentage values and also accepts a user-defined zoom value in the text portion of the combo box. When zooming, 100% indicates the actual size of the control's image. If you click Fit in the Zoom combo box, ScalableInkPicture resizes the image to the largest size possible in the pop-up form without using scroll bars.

To zoom in on an image, the SizeImage method scales the image and ink proportionally. This method first determines if the prescribed zoom increment is Fit, and if so it zooms accordingly:

//
// SizeImage
//
// Sets the image size according to the value of the Zoom combo.
//
private void SizeImage()
{
   if ((scalableInkPictureControl.Image == null) || (toolBarZoomCombo.Text.Trim() == ""))
   {
      return;
   }

   string newZoomString = toolBarZoomCombo.Text.Trim().ToUpper();

   if (newZoomString == "FIT")
   {
      // Determine the available space for the image.
      availableSize = ClientSize;
      availableSize.Height -= topBorder + bottomBorder;
      availableSize.Width -= leftBorder + rightBorder;

      hScrollBar.Visible = false;
      vScrollBar.Visible = false;
      scalableInkPictureControl.ZoomToFit();
      SetZoomCombo();
      
      fitImage = true;

      // Enable the timer in order to update the actual zoom value 
      // after the event handler finishes processing.
      zoomComboTimer.Enabled = true;
   }
   ...

If the prescribed zoom increment is Fit, the SizeImage method calls the ZoomToFit method. In this method, the outerBounds value represents the initial size and position of the control's System.Drawing.Rectangle. ScalableInkPictureControl resizes and repositions within this original size, centering the image without distortion:

/// <summary>
/// Scales the image to fit proportionally within the control's size in the IDE.
/// </summary>
public void ZoomToFit()
{
   // If the image proportions are shorter than the original proportions
   // then fill to the original width and set the height proportionately,
   // otherwise do the opposite (fill to the original height and set the
   // width proportionately).
   if (Image == null)
   {
      return;
   }

   if ((float)Image.Height / Image.Width < (float)outerBounds.Height / outerBounds.Width)
   {
      int width = outerBounds.Width;
      int height = outerBounds.Width * Image.Height / Image.Width;
      ClientSize = new Size(width, height);
      Left = outerBounds.Left;
      Top = outerBounds.Top + (outerBounds.Height - height) / 2;
   }
   else
   {
      int height = outerBounds.Height;
      int width = outerBounds.Height * Image.Width / Image.Height;
      ClientSize = new Size(width, height);
      Top = outerBounds.Top;
      Left = outerBounds.Left + (outerBounds.Width - width) / 2;
   }
}

If the prescribed zoom increment is not Fit, the SizeImage method scales the image and adds scroll bars as necessary:

   ...
   else
   {
      int newZoom;
      if (newZoomString.Substring(newZoomString.Length - 1) == "%")
      {
         newZoomString = newZoomString.Substring(0, newZoomString.Length - 1).Trim();
      }

      try
      {
         newZoom = Convert.ToInt32(newZoomString);
      }
      catch
      {
         newZoom = -1;
      }

      if ((newZoom >= minZoom) && (newZoom <= maxZoom))
      {
         // Determine the available space for the image.
         availableSize = ClientSize;
         availableSize.Height -= topBorder + bottomBorder;
         availableSize.Width -= leftBorder + rightBorder;

         newZoomString = newZoom.ToString() + "%";
         if ((toolBarZoomCombo.Text == newZoomString) && fitImage)
         {
            return;
         }
         SetZoomComboText(newZoomString);

         // Keep the scroll positions as a factor (0 to 1) so that the image
         // remains in the same position when the scrolling ranges change.
         float hScrollFactor = (float)hScrollBar.Value / hScrollBarMaximum;
         float vScrollFactor = (float)vScrollBar.Value / vScrollBarMaximum;

         scalableInkPictureControl.Width = scalableInkPictureControl.Image.Width * newZoom / 100;
         scalableInkPictureControl.Height = scalableInkPictureControl.Image.Height * newZoom / 100;

         // Set scrollbar ranges and visibilities.
         if (scalableInkPictureControl.Width > availableSize.Width)
         {
            if (scalableInkPictureControl.Height > availableSize.Height)
            {
               availableSize.Width -= vScrollBar.Width;
            }
            hScrollBar.Maximum = Convert.ToInt32(scalableInkPictureControl.Width - availableSize.Width);
            hScrollBar.Value = Convert.ToInt32(hScrollBar.Maximum * hScrollFactor);
            hScrollBar.SmallChange = hScrollBar.Maximum / 100 + 1;
            hScrollBar.LargeChange = hScrollBar.Maximum / 10 + 1;
            hScrollBar.Maximum += hScrollBar.LargeChange - 1;
            hScrollBar.Visible = true;
            availableSize.Height -= hScrollBar.Height;
         }
         else
         {
            // Ensure that Value / (Maximum - LargeChange + 1) will be 50% next time.
            hScrollBar.Maximum = 100;
            hScrollBar.LargeChange = 1;
            hScrollBar.Value = 50;
            hScrollBar.Visible = false;
         }

         if (scalableInkPictureControl.Height > availableSize.Height)
         {
            vScrollBar.Maximum = Convert.ToInt32(scalableInkPictureControl.Height - availableSize.Height);
            vScrollBar.Value = Convert.ToInt32(vScrollBar.Maximum * vScrollFactor);
            vScrollBar.SmallChange = vScrollBar.Maximum / 100 + 1;
            vScrollBar.LargeChange = vScrollBar.Maximum / 10 + 1;
            vScrollBar.Maximum += vScrollBar.LargeChange - 1;
            vScrollBar.Visible = true;
         }
         else
         {
            // Ensure that Value / (Maximum - LargeChange + 1) will be 50% next time.
            vScrollBar.Maximum = 100;
            vScrollBar.LargeChange = 1;
            vScrollBar.Value = 50;
            vScrollBar.Visible = false;
         }
      }
      else
      {
         SetZoomCombo();
      }
   }

   EnableOrDisableHand();
   PositionImage();
}

When ScalableInkPictureControl resizes, its corresponding InkOverlay must also resize in order to transform the Stroke objects. To do this, the ScalableInkPictureControl.OnResize event handler calls the SetViewTransform method:

/// <summary>
/// Adjusts the size of the image and ink after the control has been resized.
/// </summary>
public void SetViewTransform()
{
   if (ClientSize.Height == 0)
   {
      // The window is probably minimized.
      return;
   }

   float scale = (float)ClientSize.Height / Image.Height;
   Matrix m = new Matrix();
   m.Scale(scale, scale);
   Renderer.SetViewTransform(m);
}

The InkOverlay.Renderer object uses SetViewTransform to scale ink. This method also scales the pen tip and highlighter brush, and ensures that the ink and the pen tip scale in tandem with the control's image.

Panning

The pop-up ScalableInkPictureForm enables scroll bars when its image is zoomed to a size that does not fit in the form's available space. Scroll bars are enabled in the SizeImage method, shown previously.

Windows Forms scroll bars do not scroll all the way to their Maximum values. They only scroll as far as the Maximum minus the LargeChange plus one. ScalableInkPictureForm compensates in the SizeImage method by adjusting the Maximum values accordingly:

hScrollBar.Maximum += hScrollBar.LargeChange - 1;
...
vScrollBar.Maximum += vScrollBar.LargeChange - 1;

With the scroll bars properly set, you can pan over the image and its ink. To do so, click the Pan Image toolbar button (). While you pan, the cursor changes to an image of a hand and you use the left mouse button to drag (pan) the image up, down, left, and right within the form. The MouseDown, MouseMove, and MouseUp events trigger repositioning of the ScalableInkPictureControl:

//
// scalableInkPictureControl_MouseDown
// 
// Handles the scalableInkPictureControl.MouseDown event.
//
// Parameters:
//  sender - The source scalableInkPictureControl object for this event.
//  e - The EventArgs object that contains the event data.
//
private void scalableInkPictureControl_MouseDown(object sender, MouseEventArgs e)
{
   if (toolBarHand.Pushed)
   {
      mouseX = e.X + scalableInkPictureControl.Left;
      mouseY = e.Y + scalableInkPictureControl.Top;
      mouseIsDown = true;
   }
}

//
// scalableInkPicture_MouseMove
// 
// Handles the scalableInkPicture.MouseMove event.
//
// Parameters:
//  sender - The source scalableInkPicture object for this event.
//  e - The MouseEventArgs object that contains the event data.
//
private void scalableInkPicture_MouseMove(object sender, MouseEventArgs e)
{
   HandScroll(e);
}

The MouseMove event calls the HandScroll method to map the mouse movements into increments for relocating the ScalableInkPictureControl:

//
// HandScroll
// 
// Pans the image in response to a mouse event.
//
// Parameters:
//  e - The MouseEventArgs object that contains the event data.
//
private void HandScroll(MouseEventArgs e)
{
   if (mouseIsDown)
   {
      int newMouseX = e.X + scalableInkPictureControl.Left;
      int newMouseY = e.Y + scalableInkPictureControl.Top;
      int dx = mouseX - newMouseX;
      int dy = mouseY - newMouseY;
      mouseX = newMouseX;
      mouseY = newMouseY;
      if ((dx != 0) || (dy != 0))
      {
         SetScroll(dx, dy);
      }
   }
}

In turn, the HandScroll method calls the SetScroll method to set the Value properties of the scroll bars. The math in this method ensures that the scroll bars do not go below their Minimum or above their Maximum values:

/// <summary>
/// Sets the position of the image and the positions of the scroll bars.
/// </summary>
/// <param name="dx">Change in the x direction.</param>
/// <param name="dy">Change in the y direction.</param>
public void SetScroll(int dx, int dy)
{
   // Move, but stay within bounds.
   if (hScrollBar.Visible)
   {
      hScrollBar.Value =
         Math.Min(
            Math.Max(
               hScrollBar.Minimum,
               hScrollBar.Value + dx),
            hScrollBarMaximum);
   }
   
   if (vScrollBar.Visible)
   {
      vScrollBar.Value =
         Math.Min(
            Math.Max(
               vScrollBar.Minimum,
               vScrollBar.Value + dy),
            vScrollBarMaximum);
   }
   
   PositionImage();
}

As its final task, the SetScroll method calls the PositionImage method to position the control horizontally and vertically based on the current Value of each of the scroll bars:

//
// PositionImage
//
// Positions the image according to the current values of the scroll bars.
//
private void PositionImage()
{
   if (hScrollBar.Visible)
   {
      scalableInkPictureControl.Left = leftBorder 
         + availableSize.Width - scalableInkPictureControl.Width 
         + hScrollBarMaximum - hScrollBar.Value;
   }
   else
   {
      // No scroll bar is visible, so center the image horizontally.
      scalableInkPictureControl.Left = leftBorder 
         + (availableSize.Width / 2) 
         - (scalableInkPictureControl.Width / 2);
   }
   
   if (vScrollBar.Visible)
   {
      scalableInkPictureControl.Top = topBorder 
         + availableSize.Height - scalableInkPictureControl.Height 
         + vScrollBarMaximum - vScrollBar.Value;
   }
   else
   {
      // No scroll bar is visible, so center the image vertically.
      scalableInkPictureControl.Top = topBorder 
         + (availableSize.Height / 2)
         - (scalableInkPictureControl.Height / 2);
   }
}

Printing

All ScalableInkPictureControl print handling is done after printing is requested, and not during the construction of the control. This prevents unused print code from running when the control loads on a Web page, potentially creating associated security problems.

The Print method initializes a PrintDocument class and the PrintPage event:

if (!printInitialized)
{
   printDocument = new System.Drawing.Printing.PrintDocument();
   printDocument.PrintPage += new System.Drawing.Printing.PrintPageEventHandler(printDocument_PrintPage);
   printInitialized = true;
}

Next, the Print method initializes the Print dialog and its settings, displays the dialog, and then prints the document using the specified settings:

PrintDialog printDlg = new PrintDialog();
printDlg.PrinterSettings = new System.Drawing.Printing.PrinterSettings();

if (printDlg.ShowDialog() == DialogResult.OK)
{
   printDocument.PrinterSettings = printDlg.PrinterSettings;
   printDocument.Print();
}

The PrintPage event handler performs the actual printing:

//
// printDocument_PrintPage
//
// Handles the printDocument.PrintPage event.
//
// Parameters:
//  sender - The source printDocument object for this event.
//  e - The EventArgs object that contains the event data.
//
private void printDocument_PrintPage(object sender, System.Drawing.Printing.PrintPageEventArgs e)
{
   Rectangle printBounds = e.MarginBounds;
   
   if ((float)printBounds.Width / printBounds.Height > (float)Image.Width / Image.Height)
   {
      int width = printBounds.Height * ClientSize.Width / ClientSize.Height;
      printBounds.X += printBounds.Width / 2 - width / 2;
      printBounds.Width = width;
   }
   else
   {
      int height = printBounds.Width * ClientSize.Height / ClientSize.Width;
      printBounds.Y += printBounds.Height / 2 - height / 2;
      printBounds.Height = height;
   }

   // Create an image using Graphics buffer object.
   Bitmap imageForPrinting = new Bitmap(printBounds.Width, printBounds.Height);
   using (Graphics graphicsImage = Graphics.FromImage(imageForPrinting))
   using (DibGraphicsBuffer dib = new DibGraphicsBuffer())
   // Create temporary screen Graphics.
   using (Graphics graphicsForm = CreateGraphics())
   // Create temporary Graphics from the Device Independent Bitmap.
   using (Graphics graphicsTemp = dib.RequestBuffer(graphicsForm, printBounds.Width, printBounds.Height))
   {
      graphicsTemp.Clear(Color.White);

      Rectangle printRect = new Rectangle(0, 0, printBounds.Width, printBounds.Height);
                  
      // Draw the image.
      graphicsTemp.DrawImage(Image, printRect);

      // Draw the strokes onto the temporary Graphics,
      // first translating the Renderer by the control's location.

      Matrix oldViewTransform = null;
      Renderer.GetViewTransform(ref oldViewTransform);

      float scale = (float)printBounds.Height / ClientSize.Height;
      Matrix printViewTransform = new Matrix();
      printViewTransform.Scale(scale, scale);
      printViewTransform.Multiply(oldViewTransform);

      Renderer.SetViewTransform(printViewTransform);
      Renderer.Draw(graphicsTemp, Ink.Strokes);
      Renderer.SetViewTransform(oldViewTransform);

      // Use the buffer to paint onto the final image.
      dib.PaintBuffer(graphicsImage, 0, 0);

      // Draw this image onto the printer graphics,
      // adjusting for margins.
      e.Graphics.DrawImage(imageForPrinting, printBounds.Left, printBounds.Top); 
   }
}

Serialization

To transport an image and ink associated with ScalableInkPicture to and from a client-side Web form, a database record, a file, or both items must be serialized and de-serialized.

The ImageString property gets the serialized image by using the GetImageString method:

/// <summary>
/// Gets or sets the serialized image.
/// </summary>
/// <remarks>
/// Very large or very small images are not supported by this sample, 
/// so consider checking for a 'reasonable' image size here.
/// </remarks>
[DefaultValue(""), Category("Ink Control"), Description("The serialized ink annotations.")]
public string ImageString
{
   get
   {
      return GetImageString(Image);
   }
   set
   {
      Image image = null;
      if ((value != null) && (value != ""))
      {
         using (MemoryStream memory = new MemoryStream(Convert.FromBase64String(value)))
         {
            image = System.Drawing.Image.FromStream(memory);
         }
      }
      Image = image;
   }
}

//
// GetImageString
//
// Serializes an image.
//
// Parameters:
//  image - Image to serialize.
//
private string GetImageString(Image image)
{
   using (MemoryStream memory = new MemoryStream())
   {
      System.Drawing.Imaging.ImageFormat format = System.Drawing.Imaging.ImageFormat.Jpeg;
      image.Save(memory, format);
      string imageString = Convert.ToBase64String(memory.ToArray());
      return imageString;
   }
}

In a similar fashion, the InkString property gets and sets the serialized ink by using the GetInkString method.

The usefulness of serialized values for ScalableInkPicture becomes apparent when using the control on the Web.

ScalableInkPicture on the Web

A managed user control like ScalableInkPicture can be used in both Windows Forms and Web Forms without additional programming. To reference the control from HTML, use the <object> tag as follows:

<object id="sip" style="width:600px; height:400px"
   classid="Microsoft.TabletPC.Samples.ScalableInkPicture.dll#Microsoft.TabletPC.Samples.ScalableInkPicture.ScalableInkPictureControl" VIEWASTEXT>
</object>

A Web page hosting ScalableInkPicture should check to ensure that it's running on a Tablet PC. Internet Explorer on Windows XP Tablet PC Edition 2005 places the string "Tablet PC 1.7" into the user agent value, enabling the sample code on the server to dynamically check for a Tablet PC. To support more than one version of the Tablet PC platform, expand the user agent test. In the sample code, the if statement for the UserAgent test is commented out to support working with the source code on a non-Tablet computer. Remove the comment prefix to enforce only Tablet PC users:

//   if (Request.UserAgent.IndexOf("Tablet PC 1.7") >= 0)
{
   sipWrapper = new HtmlGenericControl();
   sipWrapper.ID = "sipWrapper";
   sipWrapper.InnerHtml =
"<object id=\"sip\" style=\"width:600px; height:400px\" " +
"classid=\"Microsoft.TabletPC.Samples.ScalableInkPicture.dll#Microsoft.TabletPC.Samples.ScalableInkPicture.ScalableInkPictureControl\" VIEWASTEXT>" +
"</object>";

FindControl("webSIPSampleForm").Controls.Add(sipWrapper);
}

Managing Web permissions for managed user controls can be cumbersome and sometimes requires an installation process. Generally, the goal is to create components that require only standard Web permissions. However, the Tablet PC Platform's Security And Trust topic notes that "full trust is required when a derived class inherits a class or overrides a method from the Tablet PC SDK." This requirement for full trust does not allow a derived control to run with the security normally granted to Websites by the Internet security zone. To avoid this security challenge, ScalableInkPicture does not derive from InkPicture. Instead, the WebInkPicture base class for the control derives from PictureBox and then adds an InkOverlay, providing essentially the same functionality as InkPicture but without the related security issues.

You can learn more about control permissions in the .NET Framework by using the mscorcfg utility, as documented in .NET Framework Configuration Tool. To start the .NET Framework Configuration Tool, click Start, click Run, and then type

mmc %windir%\Microsoft.NET\Framework\v1.1.4322\mscorcfg.msc.

ScalableInkPicture avoids the need for additional permissions because:

  • ScalableInkPicture does not derive from a class that requires additional permissions.
  • All abilities requiring additional permissions are removed from the control's constructor and placed into run-time code.
  • The control has properties to enable or disable features that require additional permissions. In the case of ScalableInkPicture, these include the PrintEnabled and ClipboardEnabled properties.
  • The control hides disabled functionality from the user interface, so that users are unaware of features that may cause security violation errors. For example, ScalableInkPicture removes the Cut, Copy, and Paste tools from the toolbar on its pop-up form when ClipboardEnabled is set to false.

To disable functionality that may generate security violation errors, the Web sample form disables printing and the Clipboard at run-time:

function OnLoad()
{
   ...
   webSIPSampleForm.all["sipWrapper"].all["sip"].ClipboardEnabled = false;
   webSIPSampleForm.all["sipWrapper"].all["sip"].PrintEnabled = false;
}

The image and ink to be displayed are provided to the browser by using two hidden input controls on the Web form. These controls contain the serialized text that represents the original binary objects:

<input id="imageString" type="hidden" name="imageString" runat="server">
<input id="inkString" type="hidden" name="inkString" runat="server">

The Page_Load event on the server initializes the image string:

// Serialize the image and place it in a hidden text
// field on the form.
imageString.Value = ImageSerializer.GetImageString(image);

The OnLoad event handler in the client-side HTML brings the serialized image to the ScalableInkPicture.ImageString property:

<body MS_POSITIONING="GridLayout" onload="OnLoad()">

...

function OnLoad()
{
   ...
   webSIPSampleForm.all["sipWrapper"].all["sip"].ImageString = webSIPSampleForm.imageString.value;

Because printing is disabled when ScalableInkPicture is used on the Web, the user must be provided with a printer-friendly version of the control's contents. The Website code provides a Show Flattened Image button to accomplish this. When a user click this button, the server handles the client postback by creating a flattened JPEG format image and returning it to a Web page, suitable for printing:

// Flatten the ink into the image.
ScalableInkPictureControl scalableInkPictureControl = new ScalableInkPictureControl();
scalableInkPictureControl.Image = image;
scalableInkPictureControl.InkString = inkString.Value;
scalableInkPictureControl.MergeInkIntoImage(scalableInkPictureControl.Image.Width, scalableInkPictureControl.Image.Height);
image = scalableInkPictureControl.Image;

// Save the flattened image to a file that the client can fetch.
string flattenedImageFileName = Guid.NewGuid().ToString() + ".jpg";
Context.Session["flattenedImageFileName"] = flattenedImageFileName;
string flattenedImageFilePath = Page.MapPath(flattenedImageFileName);
image.Save(flattenedImageFilePath, ImageFormat.Jpeg);

Response.Redirect("WebSIPFlattenedImage.aspx");

Figure 5 shows the Web page from Figure 3 with the ink and image flattened to JPEG image format, suitable for printing or saving.

Figure 5. Print-ready JPEG image created from ScalableInkPicture contents

Extending ScalableInkPicture

You can extend ScalableInkPicture to provide customized functionality for your enterprise or your unique application scenarios:

  • Richer object model. Expose more properties, methods, and events to provide additional design-time and run-time functionality. Examples include modifying the size of the pen tip, providing a larger color palette for the pen and highlighter, and adding the ability to place the image with flattened ink directly on the Clipboard.
  • Add more keyboard shortcuts. Currently, the arrow keys, Page Up, and Page Down support basic scrolling operations. Additional keyboard shortcuts can be enabled to support tool selection, zooming, and other functionality.
  • Multiple undo/redo. You can add an array for activity history to enable multiple undo and redo steps. Currently, ScalableInkPicture supports only a single level of undo.

Conclusion

The ScalableInkPicture user control provides a rich, out-of-the-box user experience for annotating images with ink. The control mimics the functionality of InkPicture, but extends this functionality to provide support for in-place and pop-up editing, zooming and scrolling, printing, and Web safety. Use the ScalableInkPicture source code as a starting point to provide pen-based image annotation capabilities in your Tablet PC applications.

Biography

Leszynski Group is a solution provider and independent software developer specializing in development tools, Tablet PC applications, and mobilized databases. Leszynski Group is a regular contributor of MSDN content, and the creator of popular utilities such as the Snipping Tool and Physics Illustrator.

Show:
© 2014 Microsoft