Export (0) Print
Expand All

Capturing Raw Images When Using WPF



This article describes how to capture and persist a raw image from the Microsoft Surface screen by using the Presentation layer of the Microsoft Surface SDK and elements of the Core layer and the System.Drawing assembly. The Microsoft Windows Presentation Foundation (WPF) enables you to create and use custom graphics, and the System.Drawing assembly includes methods for handling bitmap images.

The sample application in this topic creates a SurfaceWindow control that contains a single SurfaceButton control. When you place an item on the Microsoft Surface screen and press the button, an image of the item is captured and saved. The Microsoft Surface screen does not indicate that the image has been processed, but the button glows when you press it to indicate that the application responded.

Raw image captured by this sample

An image captured and saved to a file using the following code

The captured image is saved to a local Temp.bmp file, and the sample application is immediately ready to capture another image. The file name of the captured image is always the same (Temp.bmp), so previously saved images are overwritten every time that you press the button.

noteNote
You must have a device made for Surface to run the code that is used in this article. You cannot run this code by using a separate workstation and Input Simulator.

Creating the Project

  1. Open Microsoft Visual C# 2010 Express Edition (or Microsoft Visual Studio 2010), and then create a new project by using the Surface Application (WPF) template. (For more information about how to create this project, see Creating the Visual Studio Project for the WPF Quick Start Application.)

  2. Enter CaptureRawImageWithWPF as the name of the project.

  3. Add a SurfaceButton control to the Grid element in the SurfaceWindow1.xaml file. When you are finished, the SurfaceWindow1.xaml file should contain the following markup.

    <s:SurfaceWindow x:Class="RawImageInWPF.SurfaceWindow1"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:s="http://schemas.microsoft.com/surface/2008"
        Title="RawImageInWPF"
        Name="mainWindow"
        >
    
      <Grid>
        
        <!-- <ADDED> --><s:SurfaceButton Height="60" Width="90"HorizontalAlignment="Center"VerticalAlignment="Bottom"Click="SurfaceButton_Click"Content="Get Image"/><!-- </ADDED> -->
          
      </Grid>
        
    </s:SurfaceWindow>
    

Adding References

  1. In Solution Explorer, right-click References, and add references to the Microsoft.Surface.Core and System.Drawing assemblies.

  2. In the SurfaceWindow1.xaml.cs file, add the following using statements.

    //<ADDED>
    using System.Runtime.InteropServices;
    using Microsoft.Surface.Core;
    using System.Drawing;
    using System.IO;
    //</ADDED>
    

Implementing the Code in SurfaceWindow1.xaml.cs

  1. Add the following class-level private fields.

            //<ADDED>
            // Private fields.
            Microsoft.Surface.Core.TouchTarget touchTarget;
            IntPtr hwnd;
            private byte[] normalizedImage;
            private Microsoft.Surface.Core.ImageMetrics normalizedMetrics;
            System.Drawing.Imaging.ColorPalette pal;
            bool imageAvailable;
            //<ADDED>
    
  2. Modify the default constructor to instantiate the TouchTarget object, assign the FrameReceived event handler, enable input, and enable raw images.

    The modified constructor should look like the following code.

            public SurfaceWindow1()
            {
                InitializeComponent();
    
                // Add handlers for window availability events
                AddWindowAvailabilityHandlers();
    
                //<ADDED>InitializeSurfaceInput();//</ADDED>
            }
    
            //<ADDED>private void InitializeSurfaceInput(){// Get the hWnd for the SurfaceWindow object after it has been loaded.hwnd = new System.Windows.Interop.WindowInteropHelper(this).Handle;touchTarget = new Microsoft.Surface.Core.TouchTarget(hwnd);// Set up the TouchTarget object for the entire SurfaceWindow object.touchTarget.EnableInput();EnableRawImage();// Attach an event handler for the FrameReceived event.touchTarget.FrameReceived += new EventHandler<FrameReceivedEventArgs>(OnTouchTargetFrameReceived);}//</ADDED>
    
  3. Add the event handler for the Microsoft.Surface.Core.TouchTarget.FrameReceived event. This handler is called when the button that you created earlier is pressed. The image is retrieved by using the TryGetRawImage method from the FrameReceivedEventArgs class.

            //<ADDED>
            private void OnTouchTargetFrameReceived(object sender, Microsoft.Surface.Core.FrameReceivedEventArgs e)
            {
                imageAvailable = false;
                int paddingLeft,
                      paddingRight;
                if (normalizedImage == null)
                {
                    imageAvailable = e.TryGetRawImage(Microsoft.Surface.Core.ImageType.Normalized,
                        Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Left,
                        Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Top,
                        Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Width,
                        Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Height,
                        out normalizedImage,
                        out normalizedMetrics,
                        out paddingLeft,
                        out paddingRight);
                }
                else
                {
                    ;
                    imageAvailable = e.UpdateRawImage(Microsoft.Surface.Core.ImageType.Normalized,
                         normalizedImage,
                         Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Left,
                         Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Top,
                         Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Width,
                         Microsoft.Surface.Core.InteractiveSurface.PrimarySurfaceDevice.Height);
                }
            }
            //</ADDED>
    
    
  4. Modify the OnApplicationActivated and OnApplicationDeactivated event handlers to match the following code. By disabling images when the application is deactivated, the overall performance of the Microsoft Surface unit improves.

            private void OnWindowInteractive(object sender, EventArgs e)
            {
                //TODO: enable audio, animations here
    
                //<ADDED>// Enable a normalized image to be obtained.touchTarget.EnableImage(ImageType.Normalized);//</ADDED>
    
            }
    
            private void OnWindowNoninteractive(object sender, EventArgs e)
            {
                //TODO: Disable audio here if it is enabled
    
                //TODO: optionally enable animations here
    
                //<ADDED>// If the application is deactivated before a raw image is// captured, make sure to disable core raw images for performance reasons.touchTarget.DisableImage(Microsoft.Surface.Core.ImageType.Normalized);//</ADDED>
    
            }
    
            private void OnWindowUnavailable(object sender, EventArgs e)
            {
                //TODO: disable audio, animations here
    
                //<ADDED>// If the application is deactivated before a raw image is// captured, make sure to disable core raw images for performance reasons.touchTarget.DisableImage(Microsoft.Surface.Core.ImageType.Normalized);//</ADDED>
    
            }
    
    

Converting and Saving the Image

During the FrameReceived event, the normalizedImage byte array is populated with raw image data. When you press the SurfaceButton object (which is defined in SurfaceWindow1.xaml), the normalizedImage array is converted to a Bitmap object. The Palette property of the Bitmap object is then changed to grayscale, and the bitmap image is saved to disk.

As the final step in building the sample application, add the event handler method for the SurfaceButton.Click event along with the utility methods for doing the grayscale conversion, enabling and disabling raw images. These methods are presented below.

        //<ADDED>
        private void SurfaceButton_Click(object sender, RoutedEventArgs e)
        {
            if (imageAvailable)
            {
                // Stop processing raw images so normalizedImage 
                // is not changed while it is saved to a file.
                DisableRawImage();

                // Copy the normalizedImage byte array into a Bitmap object.

                GCHandle h = GCHandle.Alloc(normalizedImage, GCHandleType.Pinned);
                IntPtr ptr = h.AddrOfPinnedObject();
                Bitmap imageBitmap = new Bitmap(normalizedMetrics.Width,
                                      normalizedMetrics.Height,
                                      normalizedMetrics.Stride,
                                      System.Drawing.Imaging.PixelFormat.Format8bppIndexed,
                                      ptr);

                // The preceding code converts the bitmap to an 8-bit indexed color image. 
                // The following code creates a grayscale palette for the bitmap.
                Convert8bppBMPToGrayscale(imageBitmap);


                // The bitmap is now available to work with 
                // (such as, save to a file, send to a processing API, and so on).
                if (File.Exists("temp.bmp"))
                {
                    File.Delete("temp.bmp");
                }

                imageBitmap.Save(@"temp.bmp", System.Drawing.Imaging.ImageFormat.Bmp);
               

                imageAvailable = false;

                // Re-enable collecting raw images.
                EnableRawImage();

            }
        }

        private void Convert8bppBMPToGrayscale(Bitmap bmp)
        {
            if (pal == null) // pal is defined at module level as --- ColorPalette pal;
            {
                pal = bmp.Palette;
                for (int i = 0; i < 256; i++)
                {
                    pal.Entries[i] = System.Drawing.Color.FromArgb(i, i, i);
                }
            }

            bmp.Palette = pal;
        }

        private void EnableRawImage()
        {
            touchTarget.EnableImage(Microsoft.Surface.Core.ImageType.Normalized);
            touchTarget.FrameReceived += OnTouchTargetFrameReceived;
        }

        private void DisableRawImage()
        {
            touchTarget.DisableImage(Microsoft.Surface.Core.ImageType.Normalized);
            touchTarget.FrameReceived -= OnTouchTargetFrameReceived;
        }
        //</ADDED>

When you run your sample application, put your hand on the interactive surface device, and then press the button to capture the image. The image that Surface sees is saved to a file named temp.bmp in the same folder as your executable (.exe) file.

Did you find this information useful? Please send us your suggestions and comments.

© Microsoft Corporation. All rights reserved.
Show:
© 2015 Microsoft