Using media extensions
Media extensions add functionality to the media pipeline in your Windows Store app. Some possible extensions are audio and video effects, capture effects, transcoding, Digital Rights Management (DRM), support for new decoders and encoders, or a custom capture sink, to name a few. Some extensions are included in the system, such as the VideoStabilization effect and the DRM MediaProtectionManager. Functionality not included in the system can be added using custom media extensions.
For an in-depth sample that uses custom media extensions, see the Media Extension Sample. This sample shows how to create and enable a custom video gray-scale effect, a custom decoder, and how to use the built in VideoStabilization effect.
Another sample that uses media extensions is the Real-Time communication sample.
The Media Foundation Pipeline consists of Media Sources, Media Foundation Transforms, such as video effects, encoders and decoders, and Media Sinks. There are a number of ways to add effects and extensions to a Windows Store app. The MediaExtensionManager allows you to replace the media source and encoders and decoders. Methods on the playback, capture, and transcode objects lets you add and remove effects. The MediaCapture class allows you to add a custom sink.
How to add and remove effects to audio and video playback objects depends on the programming language you are using.
To register scheme handlers, byte-stream handlers, audio encoders, audio decoders, video encoders, and video decoders, use the MediaExtensionManager.
DRM is supported through the MediaProtectionManager.
The Windows Runtime provides a built in effect for stabilizing shaky video, such as video captured from a hand held camera.
You can use custom media extension to add new functionality to the media pipeline. For example, you could create a custom video effect to convert the video to grayscale, as in the Media extension sample. Or create a custom extension to add support for video and audio formats that are not natively supported by the system.
Creating a custom extension consists of two parts. The first is to write the Media Foundation component and the second is to wrap this component in a Windows Runtime media extension object. For an in-depth sample that creates a number of different custom media extensions, see the Media extension sample.
A Media Extension consists of a hybrid object that implements both Component Object Model (COM) and Windows Runtime interfaces. The COM interfaces interact with the Microsoft Media Foundation pipeline. The Windows Runtime interfaces activate the component and interact with the Windows Store app.
In most situations, it is recommended that you use Visual C++ with Component Extensions (C++/CX ) to interact with the Windows Runtime. But in the case of hybrid components that implement both COM and Windows Runtime interfaces, such as Media Extensions, this is not possible. C++/CX can only create Windows Runtime objects. So, for hybrid objects it is recommended that you use Windows Runtime C++ Template Library to interact with the Windows Runtime. Be aware that Windows Runtime C++ Template Library has limited support for implementing COM interfaces.
Be aware that custom extensions are only accessible to the application they are created and registered in. They are not accessible to other applications.
A custom extension cannot override a Microsoft provided component that supports the same format. For example, since Microsoft provides support for MPEG-Layer 3 (MP3), an application will not be able to use a extension that is registered for MP3.
High level steps to create a custom effect.
- Create a native Media Foundation extension and implement IMediaExtension.
- Register and activate the extension.
- Add the extension in your application to the media pipeline.
For more info on creating a Media Foundation media extension in Windows Store app, see Walkthrough: Creating a Windows Store app using WRL and Media Foundation and the Media extension sample.
- Media extension sample
- Transcoding media sample
- Real-Time communication sample
- Creating a Windows Store app using WRL and Media Foundation
- Media Foundation Transforms
- How to enable effects in video tag
- How to add video stabilization