Edit

Custom Mixers

  • Article

[The component described on this page, Enhanced Video Renderer, is a legacy feature. It has been superseded by the Simple Video Renderer (SVR) exposed through the MediaPlayer and IMFMediaEngine components. To play video content you should send data into one of these components and allow them to instantiate the new video renderer. These components have been optimized for Windows 10 and Windows 11. Microsoft strongly recommends that new code use MediaPlayer or the lower level IMFMediaEngine APIs to play video media in Windows instead of the EVR, when possible. Microsoft suggests that existing code that uses the legacy APIs be rewritten to use the new APIs if possible.]

This topic describes how to write a custom mixer for the enhanced video renderer (EVR). You can use a custom mixer with either the Media Foundation EVR media sink, or the DirectShow EVR filter. For more information about mixers and presenters, see Enhanced Video Renderer.

The mixer is a Media Foundation transform (MFT) with one or more inputs (the reference stream plus the substreams) and one output. The input stream receives samples from upstream. The output stream delivers samples to the presenter. The EVR is responsible for calling IMFTransform::ProcessInput on the mixer, and the presenter is responsible for calling IMFTransform::ProcessOutput.

At a minimum, an EVR mixer must implement the following interfaces:

Interface Description
IMFTransform Provides base MFT functionality.
IMFTopologyServiceLookupClient Enables the mixer to get interfaces from the EVR.
IMFVideoDeviceID Enables the mixer to get interfaces from the EVR.
IMFAttributes Used to expose the MF_SA_D3D_AWARE attribute to the EVR.

 

Optionally, an MFT can implement any of the following interfaces:

Interface Description
IEVRTrustedVideoPlugin Required to play protected content.
IMFGetService Exposes interfaces such as IMFVideoMixerBitmap and IMFVideoProcessor to the application.
IMFQualityAdvise Enables the quality manager to adjust the video quality.
IMFVideoMixerBitmap Enables the application to mix a static bitmap onto the video.
IMFVideoPositionMapper Maps coordinates on the output video frame to coordinates on the input video frame.
IMFVideoProcessor Exposes some DXVA video processing features to the application.

 

Format negotiation with the mixer works as follows:

  1. The EVR sets the media type on the reference stream.

  2. The EVR calls IMFVideoPresenter::ProcessMessage on the presenter with the MFVP_MESSAGE_INVALIDATEMEDIATYPE message.

  3. The presenter sets the media type on the mixer's output stream.

  4. The EVR sets the media type on the substreams.

If the media type on the reference stream changes, the mixer's other media types are no longer valid. The mixer's IMFTransform::ProcessOutput method will then fail and return MF_E_TRANSFORM_STREAM_CHANGE. The presenter should not do anything at this point. The EVR will initiate the format negotiation process again.

When any input stream reaches the end of the stream, the EVR calls IMFTransform::ProcessMessage on the mixer with MFT_MESSAGE_NOTIFY_END_OF_STREAM.

The mixer sends the following events to the EVR, using the EVR's IMediaEventSink interface. This interface is documented in the DirectShow SDK documentation.

Event Description
EC_SAMPLE_NEEDED The mixer requires a new input sample.

 

The EVR might call ProcessOutput on the mixer before streaming starts. The mixer should not fail these calls. Instead, it should fill the output surface with black pixels. The mixer should continue to color-fill output samples until it receives an MFT_MESSAGE_NOTIFY_BEGIN_STREAMING message or the ProcessInput method is called. If the mixer receives an MFT_MESSAGE_NOTIFY_END_STREAMING message, it should switch back to color-fill mode.

Implementing IMFVideoDeviceID

The IMFVideoDeviceID interface contains one method, GetDeviceID, which returns a device GUID. The device GUID ensures that the presenter and the mixer use compatible technologies. If the device GUIDs do not match, the EVR fails to initialize.

The standard mixer and presenter both use Direct3D 9, with the device GUID equal to IID_IDirect3DDevice9. If you intend to use your custom presenter with the standard mixer, the presenter's device GUID must be IID_IDirect3DDevice9. If you replace both components, you could define a new device GUID.

Implementing IMFTopologyServiceLookupClient

The mixer must implement the IMFTopologyServiceLookupClient interface. Before streaming begins, the EVR calls IMFTopologyServiceLookupClient::InitServicePointers and passes in a pointer to the EVR's IMFTopologyServiceLookup interface. The mixer uses this pointer to get interface pointers from the EVR.

At a minimum, the mixer must query for the following interface:

When the EVR calls IMFTopologyServiceLookupClient::ReleaseServicePointers, the mixer must release any pointers obtained from the call to InitServicePointers.

Mixer Attributes

A mixer should support the following attributes.

Attribute Description
MF_SA_D3D_AWARE Specifies whether the mixer supports DirectX Video Acceleration (DXVA).
MF_SA_REQUIRED_SAMPLE_COUNT The number of video samples the EVR should allocate for each mixer stream. This attribute applies to individual streams; use the attribute store returned by IMFTransform::GetInputStreamAttributes.

 

Setting the Mixer on the EVR

To set a custom mixer on the EVR, call IMFVideoRenderer::InitializeRenderer. Both the DirectShow EVR filter and the EVR media sink implement this method.

EVR Activation Object. If you are using the EVR activation object, you can provide a custom mixer by setting one of the following attributes on the EVR activation object:

Attribute Description
MF_ACTIVATE_CUSTOM_VIDEO_MIXER_ACTIVATE Pointer to an activation object for the mixer. The activation object must implement the IMFActivate interface.
MF_ACTIVATE_CUSTOM_VIDEO_MIXER_CLSID CLSID of the mixer.

 

Enhanced Video Renderer