Export (0) Print
Expand All

Capturing and Rendering a Raw Image

This example shows how to capture and render a raw image that is obtained from the Microsoft Surface vision system on a device made for Surface.


Capturing an rendering a raw image

For a complete code sample project, see the RawImage Visualizer SDK sample.

To capture and render raw images in a Surface application

  1. Open Microsoft Visual C# 2010 Express Edition (or Microsoft Visual Studio 2010), and then create a new project by using the Surface Application (XNA) template.

  2. In the project Properties window, in the XNA Game Studio tab, select Use HiDef to access the complete API under Game profile.

    The HiDef profile is required to use the Alpha8 enumeration value.

  3. Initialize the application to receive raw images.

    The C# InitializeSurfaceInput method is automatically generated by the Surface Application (XNA) template. The application must call the EnableImage method and assign a new event handler for the FrameReceived event. Modify InitializeSurfaceInput to look like this:

            private void InitializeSurfaceInput()
                System.Diagnostics.Debug.Assert(Window != null && Window.Handle != IntPtr.Zero,
                    "Window initialization must be complete before InitializeSurfaceInput is called");
                if (Window == null || Window.Handle == IntPtr.Zero)
                System.Diagnostics.Debug.Assert(touchTarget == null,
                    "Surface input already initialized");
                if (touchTarget != null)
                // Create a target for surface input.
                touchTarget = new TouchTarget(Window.Handle, EventThreadChoice.OnBackgroundThread);
                //<ADDED>// Enable a normalized image to be obtained.touchTarget.EnableImage(ImageType.Normalized);// Attach an event handler for the FrameReceived event.touchTarget.FrameReceived +=new EventHandler<FrameReceivedEventArgs>(OnFrameReceived);//</ADDED>

    This code example receives a normalized image (which, in this case, is a grayscale image that is easy to render inside the Microsoft XNA development platform).

  4. Define what happens when a FrameReceived event occurs by declaring a byte array called normalizedImage as a member of the application class and then passing the return value of the TryGetRawImage method into the normalizedImage array.

    The code for the FrameReceived event handler should be similar to the following code example.

            // Declare global variables.
            //private SpriteBatch spriteBatch;
            private Texture2D contactSprite;
            private Vector2 spriteOrigin;
            private byte[] normalizedImage;
            private ImageMetrics imageMetrics;
            // Occurs when a new frame of contact data is available.
            public void OnFrameReceived(object sender, FrameReceivedEventArgs e)
                int paddingLeft, paddingRight;
                if (null == normalizedImage)
                        out normalizedImage, out imageMetrics,
                        out paddingLeft, out paddingRight);
                    // Update the image if there is already a normalized image.
                        ImageType.Normalized, normalizedImage,

    This code example uses the UpdateRawImage method to improve performance because this method uses the byte array that is already initialized, normalizedImage. The Update method uses the imageMetrics object to create a Texture2D object.

    If the Microsoft Surface unit has an external monitor installed, the origin might not be at (0,0) because the values are in desktop coordinates, not screen coordinates. Use PrimarySurfaceDevice values (as the preceding code shows) to get the actual coordinates of the interactive surface.

  5. Create a sprite to draw during every update by using the normalizedImage byte array and then creating a Texture2D object from this array.

    The code for the Update method should be similar to the following code example.

            protected override void Update(GameTime gameTime)
                if (ApplicationServices.WindowAvailability != WindowAvailability.Unavailable)
                    if (ApplicationServices.WindowAvailability == WindowAvailability.Interactive)
                        // TODO: Process touches, 
                        // use the following code to get the state of all current touch points.
                        // ReadOnlyTouchPointCollection touches = touchTarget.GetState();
                        //<ADDED>if (normalizedImage != null){// If there is a normalized image, get a new Texture2D// object that is the size of the raw image's height and width.contactSprite = new Texture2D(graphics.GraphicsDevice,imageMetrics.Width,imageMetrics.Height,true,SurfaceFormat.Alpha8);// Set the sprite's data with the data from normalizedImage.// The size of the data will be the same as the area of the raw image.contactSprite.SetData<Byte>(normalizedImage);}//</ADDED>
                    // TODO: Add your update logic here

    You can now draw the Texture2D sprite to the screen by using standard XNA drawing procedures, as the following code example shows. The input resolution from the vision system does not necessarily match the output resolution of the screen. The ratio between the two depends on the hardware. In the following situation, the Scale variable is used to draw the image at the physical size of the object.

    //<ADDED>private const float Scale = 4F / 3F;//</ADDED>
            protected override void Draw(GameTime gameTime)
                if (!applicationLoadCompleteSignalled)
                    // Dismiss the loading screen now that we are starting to draw
                    applicationLoadCompleteSignalled = true;
                //TODO: Rotate the UI based on the value of screenTransform here if desired
                //TODO: Add your drawing code here
                //TODO: Avoid any expensive logic if application is neither active nor previewed
                //<ADDED>if (null == spriteBatch){spriteBatch = new SpriteBatch(graphics.GraphicsDevice);}// Prepare to draw the sprite batch.spriteBatch.Begin();// Draw the sprite of RawImage.if (contactSprite != null){// Add the RawImage sprite to the batch of sprites for drawing.spriteBatch.Draw(contactSprite, spriteOrigin, null, Color.White,0f, spriteOrigin, Scale, SpriteEffects.FlipVertically, 0f);}spriteBatch.End();//</ADDED>
  6. Disable capturing raw image data when the application is deactivated. Re-enable capture when the application is activated.

    The following code enables raw image input in the WindowInteractive event handler (OnWindowInteractive in the following example). It disables raw image input in the WindowNoninteractive and WindowUnavailable event handlers.

            private void OnWindowInteractive(object sender, EventArgs e)
                //TODO: Enable audio, animations here
                //TODO: Optionally enable raw image here
                //<ADDED>// Turn raw image back on again.touchTarget.EnableImage(ImageType.Normalized);//</ADDED>
            private void OnWindowNoninteractive(object sender, EventArgs e)
                //TODO: Disable audio here if it is enabled
                //TODO: Optionally enable animations here
                //<ADDED>// If the app isn't active, there's no need to keep the raw image enabledtouchTarget.DisableImage(ImageType.Normalized);//</ADDED>
            private void OnWindowUnavailable(object sender, EventArgs e)
                //TODO: Disable audio, animations here
                //TODO: Disable raw image if it's enabled
                //<ADDED>// If the application isn't active, there's no need to keep the raw image enabled.touchTarget.DisableImage(ImageType.Normalized);//</ADDED>

Did you find this information useful? Please send us your suggestions and comments.

© Microsoft Corporation. All rights reserved.
© 2015 Microsoft