Audio and Video Overview
This overview introduces the multimedia features of Silverlight and describes how to integrate sound and video into your Silverlight pages.
This topic contains the following sections.
- MediaElement Object
- MediaElement Properties
- Controlling Media Playback Interactively
- Video Player Sample
- Timeline Markers (Synchronization Points)
- Server-Side Playlists
- Server Logs
- Digital Rights Management (DRM)
- Miscellaneous Tips and Workarounds When Working with Audio and Video
- Related Topics
Adding media to a page can be as simple as adding a MediaElement to your markup and providing a Uniform Resource Identifier (URI) to the media to play. The following example creates a MediaElement and sets its Source property to the URI of a video file. The MediaElement begins playing when the page loads.
The MediaElement object natively supports MP4, Windows Meida, and MP3 files and can be extended to support many other types of meida files. For a detailed list of the formats and protocols supported, see Supported Media Formats, Protocols, and Log Fields.
The MediaElement class provides several media-specific properties:
Stretch : Specifies how video is stretched to fill the MediaElement object. Possible values are None, Uniform, UniformToFill, and Fill. The default is Fill. For more information, see the Stretch enumeration.
In addition to its media-specific properties, MediaElement also has all the properties of a UIElement, such as Opacity and Clip. For a complete list of MediaElement properties, see the MediaElement reference page.
You can interactively control media playback by using the Play, Pause, and Stop methods of a MediaElement object. The following example defines a MediaElement object and several buttons for controlling media playback.
<Grid> <Grid.ColumnDefinitions> <ColumnDefinition Width="*" /> <ColumnDefinition Width="*" /> <ColumnDefinition Width="*"/> </Grid.ColumnDefinitions> <Grid.RowDefinitions> <RowDefinition Height="*" /> <RowDefinition Height="Auto" /> </Grid.RowDefinitions> <MediaElement x:Name="media" Source="xbox.wmv" Width="300" Height="300" Grid.Column="0" Grid.Row="0" Grid.ColumnSpan="3" /> <!-- Stops media playback.--> <Button Click="StopMedia" Grid.Column="0" Grid.Row="1" Content="Stop" /> <!-- Pauses media playback. --> <Button Click="PauseMedia" Grid.Column="1" Grid.Row="1" Content="Pause" /> <!-- Begins media playback. --> <Button Click="PlayMedia" Grid.Column="2" Grid.Row="1" Content="Play" /> </Grid>
(For more information about retrieving and manipulating Silverlight objects, see Silverlight Object Trees.)
A timeline marker is metadata associated with a particular point in a media file. These markers may be created ahead of time and stored in the media file itself as script stream samples or they can be added programmatically. They are typically used to name different scenes in a video, enable users to seek to a selected position, or provide UI scripting cues.
Silverlight for Windows PhoneYou can use timeline markers on applications that target Windows Phone OS 7.1 and later.
When the MediaElement reaches a timeline marker during playback, it raises the MarkerReached event. Your code can handle this event and use timeline markers to trigger actions. A MediaElement object's Markers property enables you to access the header-embedded markers stored in the currently playing media file. You can also use this property to add new timeline markers.
For Windows Media files, the MediaElement object does support all Windows Media technology markers (markers, metadata script commands, and separate-stream script commands); however, the collection returned from the Markers property does not include any timeline markers encoded as a separate stream in the media file, such as separate-stream script commands. If a file contains both header-embedded script commands and separate-stream script commands, this collection contains only those commands that are embedded in the file header. Separate-stream script commands trigger the MarkerReached event when encountered during media playback. However, these markers cannot be accessed ahead of time by using the Markers property.
MediaElement uses TimelineMarker objects to represent the collection of markers returned by the Markers property. The TimelineMarker object provides the following properties, which describe its time, name, and value:
Type : A string that specifies the marker's type. This value can be any user-defined string.
Text : A string that specifies the marker's value. This value can be any user-defined string.
<StackPanel Margin="40"> <StackPanel Orientation="Horizontal"> <TextBlock FontSize="12" Foreground="DarkGray">Time:</TextBlock> <TextBlock x:Name="timeTextBlock" FontSize="12" /> </StackPanel> <StackPanel Orientation="Horizontal"> <TextBlock FontSize="12" Foreground="DarkGray">Type:</TextBlock> <TextBlock x:Name="typeTextBlock" FontSize="12" /> </StackPanel> <StackPanel Orientation="Horizontal"> <TextBlock FontSize="12" Foreground="DarkGray">Value:</TextBlock> <TextBlock x:Name="valueTextBlock" FontSize="12" /> </StackPanel> <!-- The MediaElement has the MarkerReached event attached. --> <MediaElement MarkerReached="OnMarkerReached" HorizontalAlignment="Left" Source="thebutterflyandthebear.wmv" Width="300" Height="200" /> </StackPanel>
The accompanying code handles the MarkerReached event. It retrieves the Time, Type, and Text values of the timeline marker that triggered the event and displays that information in the TextBlock objects declared in the preceding markup.
Defining Media Markers
There are two ways to create media markers. For Windows Media files, an editor such as the Windows Media File Editor (installed as part of the Windows Media Encoder 9 installation) or Microsoft Expression Media Encoder can be used to create markers and save them in the media file itself or in a separate stream. For WM-DRM-encrypted content, the markers must be in a separate encrypted script stream. This is not required for PlayReady-encrypted content.
The second approach is to dynamically generate new TimelineMarker objects and add them to the MediaElement through its Markers property. These timeline markers are temporary and are not saved in the media file. Any such timeline markers you create are lost when a new media file is loaded by the MediaElement.
A server-side playlist (SSPL), as it is available for Silverlight, is a sequence of media elements (either audio or video) that allows the server administrators to control the sequence of streaming media viewed by the user as well as other behaviors. This playlist can be created statically or dynamically. A server-side playlist can be used to serve media for either on-demand or broadcast streams. For more information, see Server-Side Playlists.
For many companies, streaming media is a source of revenue. Their customers stream on-demand content and live broadcasts for a fee. These companies rely on logged information to determine not only if a customer viewed content, but how long and at what quality. You can get this information by accessing logs on the server; however, the information in these logs is determined by what information the Silverlight client sends to the server. For details on what server log fields are available with Silverlight, see Supported Media Formats, Protocols, and Log Fields.
MediaStreamSource is a piece of the Silverlight runtime that removes the influence of a media file's container, giving developers direct access to APIs for manipulating encoded elementary audio and video streams.
Why would anyone want to remove the container? For one thing, having access to elementary streams means that developers can now implement scenarios that other solutions have not necessarily provided thus far.
For another reason, having access to elementary streams allows developers to implement scenarios that the Silverlight runtime has not had a chance to implement yet. Examples of this could be, RTSP:T protocol support, SHOUTcast protocol support, seamless audio looping, ID3 v1 and ID3 v2 metadata support, and many other scenarios.
Integrating Digital Rights Management into your Silverlight applications helps you to protect and securely deliver streaming or progressive download content for cross-platform playback. Silverlight 4 supports PlayReady protected WMA and WMV content via the MediaElement or MediaStreamSource classes and supports PlayReady protected H264 and AAC content via the MediaStreamSource class. Silverlight 4 also supports the use of existing WM-DRM content libraries with PlayReady license servers. For more information, see Digital Rights Management (DRM).
Try to limit the number of MediaElement objects you have in your application at once. If you have over one hundred MediaElement objects in your application tree, regardless of whether they are playing concurrently or not, MediaFailed events may fire. The way to work around this is to add MediaElement objects to the tree as they are needed and remove them when they are not.
For better performance, try to send only the video resolution that is needed to the client. For example, try not to send 720p content to the client if you only need a 200x200 scaled version.