Multitouch as a concept requires hardware that is capable of recording touch pressure that occurs on a surface. The surface might be the screen directly (as on a tablet PC device) or a separate dedicated input device (such as a graphics tablet). Of these, the tablet PC device or similar devices with touch sensitivity directly on the display are generally most relevant to Silverlight.

Multitouch requires an environment (platform and operating system, hosting application such as a browser) that can propagate the touch input to an individual application, such as your Silverlight-based application.

Windows 7 supports multitouch input at the operating system level. This is supported in part through a message (WM_TOUCH). Already at this level, the operating system provides a promotion of multitouch messages as mouse messages, so that multitouch users can use touch and gestures to substitute as mouse moves or mouse clicks, when interacting with applications that may not be touch-aware and do all their spatial input processing through mouse events/messages. Windows 7 also coalesces the messages when appropriate.

Internet Explorer version 8 as a host is also multi-touch aware, and forwards platform multitouch messages in such a manner that plug-ins such as Silverlight can interact with multitouch within the host.

Multitouch is also supported for Silverlight in current versions of FireFox hosts running on Windows 7, and for out-of-browser applications running on Windows 7. However, multitouch input is not supported for applications running in full-screen mode.

Registering for Multitouch

Mouse event promotion exists so that multitouch users can use touch and gestures to substitute as mouse moves or mouse clicks, when interacting with applications that may not be touch-aware and do all their spatial input processing through mouse events/messages. Conceptually this is the default, because applications that predate or do not consider multitouch would not know what API to call to perform the promotion, and therefore the platform does the mouse promotion in most cases. Silverlight perpetuates the general concept of mouse event promotion for much the same reasons. Any given existing Silverlight control might have handlers for mouse events, but not specifically for multitouch events. For instance a button would be expected to act as if clicked when the user used a multitouch device to interact with it.

Mouse event promotion does have the potential for event dualism in cases where there is deliberate handling of a multitouch frame and its touch points. In Silverlight, within the body of the handler for a multitouch event, you can suspend mouse event promotion for the duration of the primary touch down. For example, if you wanted to create a touch-aware button that performed different actions based on the touch characteristics, you could suspend the promotion so that your button did not promote to the usual "click" behavior, and instead went to your discrete codepaths for how to handle the input. To do this, call SuspendMousePromotionUntilTouchUp as one of the first operations of your handler.

The exact nature of mouse promotion is not documented here, because it is a platform characteristic. Generally speaking, the mechanism is a message-to-message promotion.

Silverlight processes multitouch messages at the level of the raw message, analogous to the platform WM_TOUCH, along with access to other APIs that can capture touchpoint characteristics at that raw level that are then exposed as Silverlight APIs. Silverlight does not natively process touch to gestures, use the platform capabilities to that effect, or process WM_GESTURE. (Processing WM_GESTURE would require a registration state with the platform that Silverlight does not opt into.)

If you want to process multitouch using the gesture metaphor, your code must handle the touch events and use the API exposed under Silverlight and process into gestures, with or without using the platform API for gestures. This is not trivial.

One important difference between multitouch input and other input techniques supported in Silverlight (mouse, keyboard, stylus) is that you register for multitouch events on an application-wide basis, not by adding handlers to specific input elements (UIElement objects). This is consistent with the metaphor that Silverlight as a whole is the "application" registered with the platform.

• To assign a multitouch event handler, you assign a handler for the static event Touch..::.FrameReported. System.Windows.Input..::.Touch is a static service class that exists solely for this purpose with Touch..::.FrameReported as its only API.

• The handler you write for Touch..::.FrameReported is based on the TouchFrameEventHandler delegate.

• In a typical UI design, you might have areas of the UI that you intend to support specific multitouch actions within, and other areas where it would be better to use mouse promotion and not necessarily process the input as multitouch. To determine where the primary touch point is, you may have to evaluate the overall coordinate against the location of your multitouch-aware element, and its bounds. See GetPrimaryTouchPoint for more information and example code.

• As mentioned previously, you may want to suspend mouse event promotion as part of your TouchFrameEventHandler logic. To do this, call SuspendMousePromotionUntilTouchUp as one of the first operations.

• Touch messages as reported to Silverlight are typically combined as frames, which start with a primary "down" touch point. Sometimes you are only interested in the first touch point and the first "up" But the frame may contain other touch points and "move" actions. To access the full collection of points in a frame in your handlers, call GetTouchPoints. For a given touch point, probably the most important information is its Position.

• Other APIs expose information that in the platform API would be found in the TOUCHINPUT structure. Examples of such API are: TouchFrameEventArgs..::.Timestamp; TouchDevice..::.DirectlyOver; TouchPoint..::.Size; TouchPoint..::.TouchDevice. Depending on your scenario, you might not always need this level of information.

Reference

Concepts