Exercise 1: Build a Multi-Touch Picture-Handling Application
To understand how to manage multi-touch input, we first need to understand how to handle single (mouse-based) input. To do this, we have prepared a mouse-based picture-handling application, the multi-touch HOL starter.
Task 1 – Examining the Solution
Task 2 – Testing the Existence and Readiness of Multi-Touch Hardware
In this task, we will begin programming with multi-touch. While WPF 3.5 does not support multi-touch (multi-touch events and controls will be part of WPF 4.0), there is a way to use multi-touch in the current version. To do so, we have to use the Windows7 Integration Library Sample. This library is a sample that shows how we can use the Win32 native API within .NET code.
The Windows 7 Integration Library Sample is public on the web at http://code.msdn.microsoft.com/Project/Download/FileDownload.aspx?ProjectName=WindowsTouch&DownloadId=5038. For simplicity, these libraries are provided as a lab asset under %TrainingKitInstallDir%\MultiTouch\Assets\Win7LibSample, choosing the language of your preference (C# or VB).
Task 3 – Replacing Mouse Events with Touch Events
In this task, we will remove the mouse events and replace them with touch events so we can handle pictures with our fingers.
Task 4 – Handling More than one Picture Simultaneously
In this task, we will add multi-touch support. Each finger that touches the screen gets a unique touch-id. As long as the finger continues to touch the screen, the same touch-id will continue to be associated with the finger. When the finger leaves the screen surface, the system frees this touch-id and it can be used again by the hardware. In our example, when a finger touches a picture, its unique touch-id should be associated with the picture until the finger leaves the screen. If two or more fingers touch the screen at the same time, each of them will affect its associated picture.
When using Stylus events as touch events, the touch-id can be extracted from the Stylus event args:
WPF will fire off events for each finger that touches the screen with the associated StylusDevice.Id (touch-id).
Task 5 – Handling Pictures with Multi-Touch Manipulation
Until now, handling pictures with touch events was not much different from the capability of the mouse. In this task, we’d like to:
We already know how to dispatch the right event to the corresponding PictureTracker, yet we don’t know how to conclude the action that we need to take as a result from multiple events. This is where the Windows 7 multi-touch mechanism shines. It has a manipulation processor that consumes touch-id events and generates the proper manipulation event. All you need to do is to instantiate a manipulation processor, register for its event, and feed it with pairs of touch-id + location events.
The manipulation processor is a COM object. To use it from .NET you can use the Windows7 Integration Library sample. The ManipulationProcessor .NET wrapper class constructor gets an enumeration value that tells it which manipulation actions to report. In our case, we’d like to have them all. The processor has three events, ManipulationStarted, ManipulationCompleted, and ManipulationDelta. The ManipulationDelta is the interesting event. It provides the deltas of each factor: translate, rotate, and scale.
Task 6 – Adding a PictureTracker Cache
When the user touches a picture for the first time, the application creates a new PictureTracker instance that then creates the ManipulationProcessor COM object. Whenever the user removes the last finger (touch-id) that touched the picture, the PictureTracker instance is subject to garbage collection. This in turn results in the releasing of the underlined COM object. Analyzing the common application usage, only a few pictures are likely to be manipulated at the same time. This leads to the conclusion that we need a cache for PictureTracker instances. The cache will hold the free instances of PictureTracker. When a new PictureTracker instance is needed (on ProcessDown event), first we will try to pull an instance from the cache, and only if the cache is empty will we generate a new one. When we finish manipulating a picture, we will push the PictureTracker instance to the cache. Since ManipulationCompleted is an event of the ManipulationProcessor, we will ask the PictureTracker to handle the event and forward it to the PictureTrackerManager. This requires a new reference from the PictureTracker to its PictureTrackerManager (we use the constructor to pass the reference).
Task 7 – Adding Inertia
We are almost done. Using manipulation for scaling, translating, and rotating gives the user a natural user experience. In the real world, when you push an object and remove your hand it continues to move until the friction stops it. You can generate the same behavior for our picture objects using Inertia. The Windows7 multi-touch sub-system exposes an InertiaProcessor COM object. The InertiaProcessor can initiate the same manipulation event as the ManipulationProcessor. The Windows7 Integration Library sample provides a wrapper that ties together the Manipulation and Inertia processors. The ManipulationInertiaProcessor can replace the ManipulationProcessor and provide the additional InertiaProcessor property to expose the InertiaProcessor capabilities. To issue more events, the ManipulationInertiaProcessor needs a timer. To overcome thread UI affinity problems, we prefer to have a GUI-based timer. The Windows7 Integration Library can create such a timer for us.
When the user's last finger leaves the picture object, the ManipulationInertiaProcessor initiates the OnBeforeInertia event. Set the Inertia start parameters here. You can choose a default start velocity, or you can track the current object velocity and derive the numbers from it.