• Article

Touch Input in Silverlight

Microsoft Silverlight will reach end of support after October 2021. Learn more.

Touch input is a type of input that relies on the input concept of a touch, and potentially on multiple touches and their characteristics being interpreted as a unit (sometimes called a frame). Touch input requires hardware devices that are touch-sensitive, as well as an environment that supports the infrastructure needed to expose touch events to individual applications. This topic documents support for touch input in Silverlight and describes some key concepts.

Touch input for Windows Phone

This topic does not discuss touch input as implemented by Silverlight for Windows Phone. Silverlight for Windows Phone has a substantially different programming framework for touch input, which includes specific support for gestures and manipulations in the input API at the per-element level. For more information, see Input in Silverlight for Windows Phone.

Touch Hardware

Touch input as a concept requires hardware devices that are capable of recording touch pressure that occurs on a surface. The surface might be the screen directly (as on a tablet PC device) or a separate dedicated input device (such as a graphics tablet). Of these, the tablet PC device or similar devices with touch sensitivity directly on the display are generally most relevant to Silverlight.

Platform Requirements

Touch input requires an environment (device; platform and operating system; hosting application such as a browser) that can propagate the touch input to an individual application, such as your Silverlight-based application.

Devices that support touch input are an evolving market, and specific devices are not discussed in this topic.

Windows 7 supports touch input at the operating system level. This is supported in part through a message (WM_TOUCH). Already at this level, the operating system provides a promotion of touch messages to mouse messages. The promotion is present so that touch input users can use touch and gestures to substitute as mouse moves or mouse clicks. This is useful when interacting with applications that may not be touch-aware, and the application does all its sspatial input processing through mouse events and messages. Windows 7 also coalesces the messages when appropriate, so that applications do not have to process an overflow of intermediate messages that all generate incremental events.

Internet Explorer version 8 (and later) as a browser host is also touch aware. Internet Explorer version 8 (and later) forwards platform touch messages to plug-ins such as Silverlight that are running within Internet Explorer, such that Silverlight applications can interact with touch input.

Touch Input for Silverlight (shown for IE8 host)

Multitouch Input and Platform Input in Silverlight

Touch input is also supported for Silverlight in current versions of Firefox hosts running on Windows 7, and for out-of-browser applications running on Windows 7. However, touch input is not supported for applications running in full-screen mode.

Registering for Touch

As part of the broader platform architecture for touch input, each application that wants to receive touch messages must register its HWND (Touch API for Windows 7 includes RegisterTouchWindow for this purpose). The Silverlight 5 run time takes care of this registration step, and registers Silverlight as a runtime and all applications that use Silverlight as the runtime. Therefore it is not generally necessary to interact directly with platform code to process touch input. However, the characteristics of touch interaction within Touch API for Windows 7 and Silverlight 5 touch input are fairly specific:

  • Silverlight 5 is registered for raw touch input, not gestures. If your requirements include gestures, you have to process touch input into gestures using your own application code, within the context of Silverlight. Alternatively, you may need a larger interoperation design so that you can include a separate HWND that is touch registered for gestures from the platform, and interoperates with a Silverlight content area.

  • In general, Silverlight 5 promotes raw touch input to mouse events. (However, on a per-touchframe basis, you can disable promotion, as is described in upcoming sections of this topic.)

  • Within a host, certain gestures might be promoted by the browser host to become events other than mousedown/mousemove/mouseup.

Promotion to Mouse Events

Mouse event promotion exists so that touch input users can use touch and gestures to substitute as mouse moves or mouse clicks. Conceptually this is the default, because applications that predate or do not consider touch input would not know what API to call to perform the promotion, and therefore the platform does the mouse promotion in most cases. Silverlight 5 perpetuates the general concept of mouse event promotion for much the same reasons. Any given existing Silverlight control might have handlers for mouse events, but not specifically for touch input events. For example, a button would be expected to act as if clicked when the user used a touch input device to interact with it.

Mouse event promotion does have the potential for event dualism in cases where there is deliberate handling of a multitouch touch frame and its touch points. Within the body of the handler for a Silverlight touch input event, you can suspend mouse event promotion for the duration of the primary touch down. For example, if you wanted to create a touch-aware button that performed different actions based on the touch characteristics, you could suspend the promotion so that your button did not promote to the usual "click" behavior, and instead went to your discrete logic for how to handle the input. To do this, call SuspendMousePromotionUntilTouchUp as one of the first operations of your handler.

The exact nature of mouse promotion is not documented here, because it is a platform characteristic. Generally speaking, the mechanism is a message-to-message promotion.

Touch to Gesture

Silverlight processes touch messages at the level of the raw message, analogous to the platform WM_TOUCH, along with access to other APIs that can capture touchpoint characteristics at that raw level that are then exposed as Silverlight APIs. Silverlight does not natively process touch to gestures, use the platform capabilities to that effect, or process WM_GESTURE. (Processing WM_GESTURE would require a registration state with the platform that Silverlight 5 does not opt into.)

If you want to process touch using the gesture metaphor, your code must handle the touch events and use the API exposed under Silverlight 5 and process into gestures, with or without using the platform API for gestures. This is not trivial.

Touch API

One important difference between touch input and other input techniques supported in Silverlight (mouse, keyboard, stylus) is that you register for touch events on an application-wide basis, not by adding handlers to specific input elements (UIElement objects). This is consistent with the metaphor that Silverlight as a whole is the "application" registered with the platform.

  • To assign a touch input event handler, you assign a handler for the static event Touch.FrameReported. System.Windows.Input.Touch is a static service class that exists solely for this purpose with Touch.FrameReported as its only API.

  • The handler you write for Touch.FrameReported is based on the TouchFrameEventHandler delegate.

  • In a typical UI design, you might have areas of the UI that you intend to support specific touch actions within, and other areas where it would be better to use mouse promotion and not necessarily process the input as touch input. To determine where the primary touch point is, you may have to evaluate the overall coordinate against the location of your touch-aware element, and its bounds. See GetPrimaryTouchPoint for more information and example code.

  • As mentioned previously, you may want to suspend mouse event promotion as part of your TouchFrameEventHandler logic. To do this, call SuspendMousePromotionUntilTouchUp as one of the first operations.

  • Touch messages as reported to Silverlight 5 are typically combined as frames, which start with a primary "down" touch point. Sometimes you are only interested in the first touch point and the first "up" But the frame may contain other touch points and "move" actions. To access the full collection of points in a frame in your handlers, call GetTouchPoints. For a given touch point, probably the most important information is its Position.

  • Other APIs expose information that in the platform API would be found in the TOUCHINPUT structure. Examples of such API are: TouchFrameEventArgs.Timestamp; TouchDevice.DirectlyOver; TouchPoint.Size; TouchPoint.TouchDevice. Depending on your scenario, you might not always need this level of information.

Manipulation API in Silverlight

The Silverlight 5 UIElement class has several events related to manipulations, as well as event support classes such as ManipulationStartedEventArgs. However, these APIs and the related manipulation concepts do not have full support under Silverlight 5 at run time. To handle these events, your application must be targeting Silverlight for Windows Phone. The events exist in the Silverlight 5 assemblies in order to provide common designer support for both Silverlight for the desktop and Silverlight for Windows Phone.