• Article

March 2013

Volume 28 Number 03

DirectX - Using XAML with DirectX and C++ in Windows Store Apps

By Doug Erickson | March 2013

Since Windows Vista, DirectX has been the core graphics API for the Windows platform, enabling graphics processing unit (GPU) acceleration for all OS screen-drawing operations. However, until Windows 8, DirectX developers had to roll their own UI frameworks from the ground up in native C++ and COM, or license a middleware UI package such as Scaleform.

In Windows 8, you can bridge the gap between native DirectX and a proper UI framework with the DirectX-XAML interop feature of the Windows Runtime (WinRT). To take advantage of the API support for XAML in DirectX, you’re required to use “native” C++ (although you have access to smart pointers and the C++ component extensions). A little basic knowledge of COM helps as well, although I’ll spell out the specific interop you must perform to bring the XAML framework and DirectX operations together.

In this two-part series of articles, I’ll look at two approaches to DirectX-XAML interop: one where I draw surfaces into XAML framework elements with the DirectX graphics APIs; and one where I draw XAML controls hierarchy atop a DirectX swap chain surface.

This article discusses the first scenario, where you render to images or primitives displayed within your XAML framework.

But first, here’s a quick overview of your API options. Right now, there are three XAML types in the Windows Runtime that support DirectX interop:

  • Windows::UI::Xaml::Media::Imaging::SurfaceImage­Source (SurfaceImageSource hereafter): This type lets you draw relatively static content to a shared XAML surface using DirectX graphics APIs. The view is entirely managed by the WinRT XAML framework, which means that all presentation elements are likewise managed by it. This makes it ideal for drawing complex content that doesn’t change every frame, but less ideal for complex 2D or 3D games that update at a high frequency.
  • Windows::UI::Xaml::Media::Imaging::VirtualSurface­ImageSource (VirtualSurfaceImageSource hereafter): Like SurfaceImageSource, this uses the graphics resources defined for the XAML framework. Unlike SurfaceImage­Source, VirtualSurfaceImageSource supports logically large surfaces in an optimized, region-based way, such that DirectX only draws the regions of the surface that change between updates. Choose this element if you’re creating a map control, for example, or a large, image-dense document viewer. Again, like SurfaceImageSource, this isn’t a good choice for complex 2D or 3D games, especially ones that rely on real-time visuals and feedback.
  • Windows::UI::Xaml::Controls::SwapChainBackgroundPanel (SwapChainBackgroundPanel hereafter): This XAML control element and type allows your app to use a custom DirectX view provider (and swap chain) on top of which you can draw XAML elements, and which allows for better performance in scenarios that require very low-latency presentation or high-frequency feedback (for example, modern games). Your app will manage the DirectX device context for the SwapChainBackgroundPanel separately from the XAML framework. Of course, this means that both the SwapChainBackgroundPanel and the XAML frames aren’t synchronized with each other for refresh. You can also render to a SwapChainBackgroundPanel from a background thread.

This time, I’ll take a look at the SurfaceImageSource and Virtual­SurfaceImageSource APIs, and how you can incorporate them into your rich image and media XAML controls (SwapChainBackgroundPanel is special and gets its own article).

Note: SurfaceImageSource and VirtualSurfaceImageSource can be used from C# or Visual Basic .NET, although the DirectX rendering component must be written in C++ and compiled to a separate DLL accessed from the C# project. There are also third-party managed WinRT DirectX frameworks, such as SharpDX (sharpdx.org) and MonoGame (monogame.net), which you can use instead of SurfaceImageSource or VirtualSurfaceImageSource.

So, let’s get started. This article assumes that you understand the basics of DirectX, specifically Direct2D, Direct3D and Microsoft DirectX Graphics Infrastructure (DXGI). Of course, you know XAML and C++; this is an intermediate subject for Windows app developers. Thus girded: onward!

SurfaceImageSource and DirectX Image Composition

The Windows::UI::Xaml::Media::Imaging namespace contains SurfaceImageSource type, alongside many of the other XAML imaging types. In fact, the SurfaceImageSource type provides a way to dynamically draw to the shared surfaces of many XAML graphics and imaging primitives, effectively filling them with the contents you render with DirectX graphics calls and applying them as a brush. (Specifically, it’s an ImageSource that you use as an ImageBrush.) Think of it like a bitmap that you’re generating on the fly with DirectX, and consider that you can use this type in many places where you could apply a bitmap or other image resource.

For the purposes of this section, I’ll draw into an <Image> XAML element that contains a blank PNG image as a placeholder. I provide a height and width for the <Image> element, because this information is passed to the SurfaceImageSource constructor in my code (if I don’t provide a height and width, the content I render will be stretched to fit the <Image> tag parameters):

<Image x:Name="MyDxImage" Width="300" Height="200" Source="blank-image.png" />

In this example, my target is the <Image> tag, which will display the surface into which I’m drawing. I could use a XAML primitive as well, such as a <Rectangle> or an <Ellipse>, both of which can be filled by a SurfaceImageSource brush. This is possible because the drawings for these primitives and images are performed with DirectX by the Windows Runtime; all I’m doing is hooking up a different rendering source under the covers, as it were.

In my code, I include the following:

#include <wrl.h>
#include <wrl\client.h>
#include <dxgi.h>
#include <dxgi1_2.h>
#include <d2d1_1.h>
#include <d3d11_1.h>
#include "windows.ui.xaml.media.dxinterop.h"

These are the headers for the Windows Runtime Library (WRL), some key DirectX components and, most important, the native DirectX interop interfaces. The need for the latter to be included will become apparent soon.

I also import the corresponding libraries: dxgi.lib, d2d1.lib and d3d11.lib.

And for convenience, I’ll also include the following namespaces:

using namespace Platform;
using namespace Microsoft::WRL;
using namespace Windows::UI::Xaml::Media;
using namespace Windows::UI::Xaml::Media::Imaging;

Now, in the code, I create a type, MyImageSourceType, that inherits from the base SurfaceImageSource type and calls its constructor, as shown in Figure 1.

Figure 1 Deriving from SurfaceImageSource

public ref class MyImageSourceType sealed : Windows::UI::Xaml::
  Media::Imaging::SurfaceImageSource
{
  // ...
  MyImageSourceType::MyImageSourceType(
    int pixelWidth,
    int pixelHeight,
    bool isOpaque
  ) : SurfaceImageSource(pixelWidth, pixelHeight, isOpaque)
  {
    // Global variable that contains the width,
    // in pixels, of the SurfaceImageSource.
    m_width = pixelWidth;
    // Global variable that contains the height, 
    // in pixels, of the SurfaceImageSource.
    m_height = pixelHeight;
    CreateDeviceIndependentResources();
    CreateDeviceResources();
  }
  // ...
}

Note: You don’t need to inherit from SurfaceImageSource, although it makes things a bit easier from a code organization per­spective. You can simply instantiate a SurfaceImageSource object as a member, and use it instead. Just mentally substitute the name of your member for the object self-reference (this) in the code examples.

The CreateDeviceResources and CreateDeviceIndependent­Resources methods are user implementations that are a convenient way to logically separate the setup specific to the DirectX graphics hardware interface and the more general DirectX app-specific setup. The actions taken in both methods are essential. However, it’s good (and necessary) design to separate them, as there are times when you might want to re-create the device resources without affecting the device-independent resources, and vice versa.

CreateDeviceResources should look similar to the code in Figure 2, at least in a basic form.

Figure 2 Creating the DirectX Device-Specific Resources

// Somewhere in a header you have defined the following:
Microsoft::WRL::ComPtr<ISurfaceImageSourceNative> m_sisNative;
// Direct3D device.
Microsoft::WRL::ComPtr<ID3D11Device> m_d3dDevice;
// Direct2D objects.
Microsoft::WRL::ComPtr<ID2D1Device> m_d2dDevice;
Microsoft::WRL::ComPtr<ID2D1DeviceContext> m_d2dContext;
// ...
void MyImageSourceType::CreateDeviceResources()
{
  // This flag adds support for surfaces with a different color channel ordering
  // from the API default. It’s required for compatibility with Direct2D.
  UINT creationFlags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;
#if defined(_DEBUG)   
  // If the project is in a debug build, enable debugging via SDK Layers.
  creationFlags |= D3D11_CREATE_DEVICE_DEBUG;
#endif
  // This array defines the set of DirectX hardware feature levels this
  // app will support. Note the ordering should be preserved.
  // Don't forget to declare your application's minimum required
  // feature level in its description. All applications are assumed
  // to support 9.1 unless otherwise stated.
  const D3D_FEATURE_LEVEL featureLevels[] =
  {
    D3D_FEATURE_LEVEL_11_1,
    D3D_FEATURE_LEVEL_11_0,
    D3D_FEATURE_LEVEL_10_1,
    D3D_FEATURE_LEVEL_10_0,
    D3D_FEATURE_LEVEL_9_3,
    D3D_FEATURE_LEVEL_9_2,
    D3D_FEATURE_LEVEL_9_1,
  };
  // Create the Direct3D 11 API device object.
  D3D11CreateDevice(
    nullptr,                       
    D3D_DRIVER_TYPE_HARDWARE,
    nullptr,
    creationFlags,                 
    featureLevels,                 
    ARRAYSIZE(featureLevels),
    // Set this to D3D_SDK_VERSION for Windows Store apps.
    D3D11_SDK_VERSION,
    // Returns the Direct3D device created in a global var.
    &m_d3dDevice,                  
      nullptr,
      nullptr);
    // Get the Direct3D API device.
    ComPtr<IDXGIDevice> dxgiDevice;
    m_d3dDevice.As(&dxgiDevice);
    // Create the Direct2D device object and a
    // corresponding device context.
    D2D1CreateDevice(
      dxgiDevice.Get(),
      nullptr,
      &m_d2dDevice);
    m_d2dDevice->CreateDeviceContext(
      D2D1_DEVICE_CONTEXT_OPTIONS_NONE,
      &m_d2dContext);
    // Associate the DXGI device with the SurfaceImageSource.
    m_sisNative->SetDevice(dxgiDevice.Get());
}

At this point, I’ve created a hardware device context and tied it to a ... wait, what’s ISurfaceImageSourceNative? That’s not a WinRT type! What’s going on here?

This is the interop bit. This is where I sneak into the “Jeffries tube” of the WRL and do a little rewiring. It’s also where I get into the COM that sits behind much of the WRL.

In order to enable this interop behavior, I need to essentially plug in that DirectX source under the hood. To do so, I need to hook my type into implementation of the methods defined on the WRL-specific COM interface, ISurfaceImageSourceNative. When I’ve done that, I’ll attach the type to the <Image> element (in this example), and when the app pushes an update to the XAML framework, it’ll use my DirectX implementations of the draw calls instead of the default ones.

ISurfaceImageSourceNative is defined in the interop header I specified earlier. See what’s going on here?

Now, in my app-specific CreateDeviceIndependentResources method, I break out the COM and query for the native methods defined on SurfaceImageSource. Because these methods aren’t exposed directly, they must be obtained with a call to IUnknown::Query­Interface on the SurfaceImageSource or SurfaceImageSource-­derived type. To do this, I recast my SurfaceImageSource-derived type as IUnknown, the base interface for any COM interface (I could also cast it as IInspectable, the “base” interface for any WinRT type, which inherits from IUnknown). Then, to get a list of the ISurfaceImageSourceNative methods, I query that interface, like so:

void MyImageSourceType::CreateDeviceIndependentResources()
{
  // Query for ISurfaceImageSourceNative interface.
  reinterpret_cast<IUnknown*>(this)->QueryInterface(
    IID_PPV_ARGS(&m_sisNative));
}

(IID_PPV_ARGS is a helper macro for the WRL that retrieves an interface pointer. Very convenient! If you aren’t inheriting from SurfaceImageSource, substitute your SurfaceImageSource object’s member name for this.)

Finally, this part of the CreateDeviceResources method makes sense:

m_sisNative->SetDevice(dxgiDevice.Get());

ISurfaceImageSourceNative::SetDevice takes the configured graphics interface and couples it to the surface for any drawing operations. Note, though, that it also means I should call Create­DeviceResources after calling CreateDeviceIndependentResources at least once prior—or I won’t have a configured device to attach.

I’ve now exposed the underlying ISurfaceImageSourceNative implementation of the SurfaceImageSource type from which my MyImageSourceType type derives. I’ve effectively flipped open the hood and moved the wires to the SurfaceImageSource type, albeit to the base implementation of the draw calls and not my own. Now, I’ll implement my calls.

To do this, I implement the following methods:

  • BeginDraw: This opens the device context for drawing.
  • EndDraw: This closes the device context.

Note: I’ve chosen the method names BeginDraw and EndDraw for a sort of loose correspondence with the ISurfaceImageSourceNative methods. This pattern is for convenience and isn’t enforced.

My BeginDraw method (or other draw-initialization method I define on the derived type) must, at some point, call ISurface­ImageSourceNative::BeginDraw. (For optimizations, you can add a parameter for a sub-rectangle with the region of the image to update.) Likewise, the EndDraw method should call ISurface­ImageSourceNative::EndDraw.

The BeginDraw and EndDraw methods, in this case, could look something like the code shown in Figure 3.

Figure 3 Drawing to the DirectX Surface

void MyImageSourceType::BeginDraw(Windows::Foundation::Rect updateRect)
{   
  POINT offset;
  ComPtr<IDXGISurface> surface;
  // Express target area as a native RECT type.
  RECT updateRectNative;
  updateRectNative.left = static_cast<LONG>(updateRect.Left);
  updateRectNative.top = static_cast<LONG>(updateRect.Top);
  updateRectNative.right = static_cast<LONG>(updateRect.Right);
  updateRectNative.bottom = static_cast<LONG>(updateRect.Bottom);
  // Begin drawing - returns a target surface and an offset
  // to use as the top-left origin when drawing.
  HRESULT beginDrawHR = m_sisNative->BeginDraw(
    updateRectNative, &surface, &offset);
  if (beginDrawHR == DXGI_ERROR_DEVICE_REMOVED ||
    beginDrawHR == DXGI_ERROR_DEVICE_RESET)
  {
    // If the device has been removed or reset, attempt to
    // re-create it and continue drawing.
    CreateDeviceResources();
    BeginDraw(updateRect);
  }
  // Create render target.
  ComPtr<ID2D1Bitmap1> bitmap;
  m_d2dContext->CreateBitmapFromDxgiSurface(
    surface.Get(),
    nullptr,
    &bitmap);
  // Set context's render target.
  m_d2dContext->SetTarget(bitmap.Get());
  // Begin drawing using D2D context.
  m_d2dContext->BeginDraw();
  // Apply a clip and transform to constrain updates to the target update
  // area. This is required to ensure coordinates within the target surface
  // remain consistent by taking into account the offset returned by
  // BeginDraw, and can also improve performance by optimizing the area
  // that's drawn by D2D. Apps should always account for the offset output
  // parameter returned by BeginDraw, because it might not match the passed
  // updateRect input parameter's location.
  m_d2dContext->PushAxisAlignedClip(
    D2D1::RectF(
      static_cast<float>(offset.x), 
      static_cast<float>(offset.y), 
      static_cast<float>(offset.x + updateRect.Width),
      static_cast<float>(offset.y + updateRect.Height)), 
      D2D1_ANTIALIAS_MODE_ALIASED);
    m_d2dContext->SetTransform(
      D2D1::Matrix3x2F::Translation(
        static_cast<float>(offset.x),
        static_cast<float>(offset.y)
        )
    );
}
// End drawing updates started by a previous BeginDraw call.
void MyImageSourceType::EndDraw()
{
  // Remove the transform and clip applied in BeginDraw because
  // the target area can change on every update.
  m_d2dContext->SetTransform(D2D1::IdentityMatrix());
  m_d2dContext->PopAxisAlignedClip();
  // Remove the render target and end drawing.
  m_d2dContext->EndDraw();
  m_d2dContext->SetTarget(nullptr);
  m_sisNative->EndDraw();
}

Note that my BeginDraw method takes a Rect primitive as input, which is mapped to a native RECT type. This RECT defines the region of the screen that I intend to draw into with a corresponding SurfaceImageSource. BeginDraw, however, can only be called once at a time; I’ll have to queue up the BeginDraw calls for each SurfaceImageSource, one after the other.

Also notice that I initialize a reference to an IDXGISurface, and an offset POINT struct that contains the (x, y) offset coordinates of the RECT I’ll draw into the IDXGISurface with respect to the upper left. This surface pointer and offset are returned from ISurfaceImageSourceNative::BeginDraw to provide the IDXGISurface for drawing. Future calls in the example create a bitmap from the received surface pointer and draw into it with Direct2D calls. When ISurfaceImageSourceNative::EndDraw is called in the EndDraw overload, a completed image is the final result—an image that will be available to draw to the XAML image element or primitive.

Let’s take a look at what I’ve got:

  • A type that I derived from SurfaceImageSource.
  • Methods on my derived type that define its drawing behavior to a provided RECT on the screen.
  • The DirectX graphics resources I need to perform the drawing.
  • An association between the DirectX graphics device and the SurfaceImageSource.

What I still need is:

  • Some code that does the actual image rendering into a RECT.
  • A connection between the specific <Image> instance (or primitive) in the XAML and the SurfaceImageSource instance, to be invoked by the app.

The code for the drawing behavior is up to me, and it’s probably simplest to implement it on my SurfaceImageSource type as a specific public method that can be called from the codebehind.

The rest is easy. In the codebehind for my XAML, I add this code to my constructor:

// An image source derived from SurfaceImageSource,
// used to draw DirectX content.
MyImageSourceType^ _SISDXsource = ref new
  MyImageSourceType((int)MyDxImage->Width, (int)MyDxImage->Height, true);
// Use MyImageSourceType as a source for the Image control.
MyDxImage->Source = _SISDXsource;

And add an event handler in the same codebehind, similar to this:

private void MainPage::MyCodeBehindObject_Click(
  Object^ sender, RoutedEventArgs^ e)
{
  // Begin updating the SurfaceImageSource.
  SISDXsource->BeginDraw();
  // ... Your DirectX drawing/animation calls here ...
  // such as _SISDXsource->
  // DrawTheMostAmazingSpinning3DShadedCubeEver();
  // ...
  // Stop updating the SurfaceImageSource and draw its contents.
  SISDXsource->EndDraw();
}

(Alternatively, if I’m not deriving from SurfaceImageSource, I could place the calls to BeginDraw and EndDraw inside a method—such as DrawTheMostAmazingSpinning3DShadedCubeEver from the previous code snippet—on my drawing object.)

Now, if I’m using a XAML primitive, such as Rect or Ellipse, I create an ImageBrush and attach the SurfaceImageSource to it, like this (where MySISPrimitive is a XAML graphics primitive):

// Create a new image brush and set the ImageSource
// property to your SurfaceImageSource instance.
ImageBrush^ myBrush = new ImageBrush();
myBrush->ImageSource = _SISDXsource;
MySISPrimitive->Fill = myBrush;

And that’s it! To recap, the process in my example is:

  1. Choose a XAML imaging element, such as an Image, ImageBrush or graphics primitive (Rect, Ellipse and so on), that I intend to render into. Also, determine if the surface will provide an animated image. Place it in my XAML.
  2. Create the DirectX device and device contexts (typically Direct2D or Direct3D, or both) that will be used for drawing operations. Also, using COM, acquire a reference to the ISurfaceImageSourceNative interface that underpins the SurfaceImageSource runtime type and associate the graphics device with it.
  3. Create a type that derives from SurfaceImageSource, and which has code that calls ISurfaceImageSource::BeginDraw and ISurfaceImageSource::EndDraw.
  4. Add any specific drawing operations as methods on the SurfaceImageSource type.
  5. For Image surfaces, connect the Source property to a SurfaceImageSource type instance. For graphics primitive surfaces, create an ImageBrush and assign a SurfaceImage­Source instance to the ImageSource property, and then use that brush with the primitive’s Fill property (or any property that accepts an ImageSource or ImageBrush).
  6. Call the draw operations on the SurfaceImageSource instances from event handlers. For animated images, ensure that the frame-draw operations are interruptible.

I can use Surface­ImageSource for 2D and 3D game scenarios if the scene and shaders are simple enough. For example, a graph­ically middleweight strategy game (think “Civilization 4”) or simple dungeon crawler could render the visuals to a SurfaceImageSource.

Also, note that I can create the derived SurfaceImageSource type in C++ in a separate DLL and use that type from a different, non-C++ language projection. In this case, I could confine my renderer and methods to C++, and build my app infrastructure and codebehinds in, say, C#. Model-View-ViewModel (MVVM) rules!

Which brings us to the limitations:

  • The control that displays the SurfaceImageSource is designed for fixed-size surfaces.
  • The control that displays the SurfaceImageSource isn’t performance-optimized for arbitrarily large surfaces, especially surfaces that can be dynamically panned or zoomed.
  • The control refresh is handled by the WinRT XAML framework view provider, which occurs when the framework refreshes. For real-time, high-fidelity graphics scenarios, this can impact performance noticeably (meaning it’s not well-suited for your hot new shader-intensive intergalactic battle game).

This brings us to VirtualSurfaceImageSource (and, eventually, SwapChainBackgroundPanel). Let’s take a look at the former.

VirtualSurfaceImageSource and Interactive Control Rendering

VirtualSurfaceImageSource is an extension of SurfaceImageSource, but it’s designed for image surfaces that might be resized by the user, especially images that can be sized larger than the screen or moved partially offscreen, or that might have other images or XAML elements obscuring part or all of them. It works particularly well with apps in which the user regularly pans or zooms an image that’s potentially larger than the screen, for example, a map control or an image viewer.

The process for VirtualSurfaceImageSource is identical to that for SurfaceImageSource as presented earlier, only you use the VirtualSurfaceImageSource type instead of SurfaceImageSource, and the IVirtualImageSourceNative interface implementation instead of the ISurfaceImageSourceNative one.

That means I change my code from the prior example to:

  • Use VirtualSurfaceImageSource instead of SurfaceImage­Source. In the code samples following, I’ll have my base image source type class, MyImageSourceType, derive from VirtualSurfaceImageSource.
  • Query for the method implementation on the underlying IVirtualSurfaceImageSourceNative interface.

See Figure 4 for an example.

Figure 4 Inheriting from VirtualSurfaceImageSource

public ref class MyImageSourceType sealed : Windows::UI::Xaml::Media::Imaging::VirtualSurfaceImageSource
{
  // ...
  MyImageSourceType::MyImageSourceType(
    int pixelWidth,
    int pixelHeight,
    bool isOpaque
    ) : VirtualSurfaceImageSource(pixelWidth, pixelHeight, isOpaque)
  {
    // Global variable that contains the width, in pixels,
    // of the SurfaceImageSource.
    m_width = pixelWidth;
    // Global variable that contains the height, in pixels,
    // of the SurfaceImageSource.
    m_height = pixelHeight;
    CreateDeviceIndependentResources(); // See below.
    CreateDeviceResources(); //Set up the DXGI resources.
  }
  // ...
  void MyImageSourceType::CreateDeviceIndependentResources()
  {
    // Query for IVirtualSurfaceImageSourceNative interface.
    reinterpret_cast<IUnknown*>(this)->QueryInterface(
      IID_PPV_ARGS(&m_vsisNative));
  }
  // ...
}

Oh, and there’s one other very important difference: I must implement a callback that’s invoked whenever a “tile” (a defined rectangular region, not to be confused with Windows 8 UI tiles) of the surface becomes visible and needs to be drawn. These tiles are managed by the framework when an app creates an instance of VirtualSurfaceImageSource, and you don’t control their parameters. Rather, behind the scenes, a large image is subdivided into these tiles, and the callback is invoked whenever a portion of one of these tiles becomes visible to the user and requires an update.

To use this mechanism, I first need to implement an instantiable type that inherits from the IVirtualSurfaceUpdatesCallbackNative interface, and register an instance of that type by passing it to IVirtualSurfaceImageSource::RegisterForUpdatesNeeded, as shown in Figure 5.

Figure 5 Setting up a Callback for a VirtualSurfaceImageSource

class MyVisibleSurfaceDrawingType :
  public IVirtualSurfaceUpdatesCallbackNative
{
// ...
private:
  virtual HRESULT STDMETHODCALLTYPE UpdatesNeeded() override;
}
// ...
HRESULT STDMETHODCALLTYPE MyVisibleSurfaceDrawingType::UpdatesNeeded()
{
  // ... perform drawing here ...
}
void MyVisibleSurfaceDrawingType::Initialize()
{
  // ...
  m_vsisNative->RegisterForUpdatesNeeded(this);
  // ...
}

The drawing operation is implemented as the UpdatesNeeded method from the IVirtualSurfaceUpdatesCallbackNative interface. If a specific region has become visible, I must determine which tiles should be updated. I do this by calling IVirtualSurfaceImage­SourceNative::GetRectCount and, if the count of updated tiles is greater than zero, getting the specific rectangles for those updated tiles with IVirtualSurfaceImageSourceNative::GetUpdateRects and updating each one:

HRESULT STDMETHODCALLTYPE MyVisibleSurfaceDrawingType::UpdatesNeeded()
{
  HRESULT hr = S_OK;
  ULONG drawingBoundsCount = 0; 
  m_vsisNative->GetUpdateRectCount(&drawingBoundsCount);
  std::unique_ptr<RECT[]> drawingBounds(new RECT[drawingBoundsCount]);
  m_vsisNative->GetUpdateRects(drawingBounds.get(), drawingBoundsCount);
  for (ULONG i = 0; i < drawingBoundsCount; ++i)
  {
    // ... per-tile drawing code here ...
  }
}

I can get the VirtualSurfaceImageSource-defined parameters for these tiles as RECT objects. In the preceding example, I get an array of RECT objects for all the tiles that need updates. Then I use the values for the RECTs to redraw the tiles by supplying them to VirtualSurfaceImageSource::BeginDraw.

Again, as with SurfaceImageSource, I initialize a pointer to the IDXGISurface, and I call the BeginDraw method on IVirtualSurfaceImageSourceNative (the underlying native interface implementation) to get the current surface into which to draw. The offset, however, refers to the (x, y) offset for the target RECT, rather than the image element as a whole.

For each RECT to update, I call code that looks like Figure 6.

Figure 6 Handling Updates to the Control Size or Visibility

POINT offset;
ComPtr<IDXGISurface> dynamicSurface;
// Set offset.
// Call the following code once for each tile RECT that
// needs to be updated.
HRESULT beginDrawHR = m_vsisNative->
  BeginDraw(updateRect, &dynamicSurface, &offset);
if (beginDrawHR == DXGI_ERROR_DEVICE_REMOVED ||
  beginDrawHR == DXGI_ERROR_DEVICE_RESET)
{
  // Handle the change in the graphics interface.
}
else
{
  // Draw to IDXGISurface for the updated RECT at the provided offset.
}

Again, I can’t parallelize these calls, because the graphics inter­face can have only one operation on the UI thread at a time. I can process each tile RECT in serial, or I can call IVirtualSurface­ImageSourceNative::BeginDraw with a unioned area of all RECTs for a single draw update. This is up to the developer.

Finally, I call IVirtualSurfaceImageSourceNative::EndDraw after I update each changed tile RECT. When the last updated tile is processed, I’ll have a completely updated bitmap to provide to the corresponding XAML image or primitive, just as I did in the SurfaceImageSource example.

And that’s it! This form of DirectX-XAML interop is great when users don’t care about low-latency input for real-time 3D graphics, as they might in a detailed, real-time game. It’s also awesome for graphics-rich apps and controls and more asynchronous (read: turn-based) games.

In the follow-up article, I’ll take a look at the flipside of this approach: drawing XAML on top of the DirectX swap chain and the work needed to make the XAML framework play nice with a custom DirectX view provider. Stay tuned!

Get help building your Windows Store app!

Doug Erickson is a senior programming writer at Microsoft, working in Windows Content Services and specializing in DirectX and Windows Store game development. He hopes you’ll make lots of amazing Windows Store DirectX games and become famous.

Thanks to the following technical experts for reviewing this article: Jesse Bishop and Bede Jordan
Jesse Bishop is the program manager for XAML graphics, rendering, media and DirectX interop on the Windows UI platform team.  Since joining Microsoft in 2008 Jesse has worked on various developer platform technologies including Windows and Silverlight.

Bede Jordan is the software development lead for XAML graphics, rendering, media, DirectX interop and text on the Windows UI platform team. Since joining Microsoft in 2005, he has worked on developer UI platforms and core Windows presentation technologies.