Streaming Media Performance

  • Article

Applies To: Windows 8, Windows 8.1

The Streaming Media Performance assessment helps you assess the performance of a particular computer configuration when you stream media using Internet Explorer. You can use the assessment results to understand, compare, and improve the streaming media experience.

The Streaming Media Performance assessment uses a streaming server application that is deployed either on a local computer or on a remote server. The assessment starts Internet Explorer and plays back the media content from start to finish or to a specified time. Then, Internet Explorer closes and results are generated. The following graphic illustrates the assessment process.

The focus of the assessment is on video playback using Internet Explorer. The assessment mimics a user who is watching a movie and reports any glitches that it detects. The assessment plays video content from several workloads that range from low resolution to high resolution. When it is run on a single computer, the process uses HTML 5.0 pages that are available in the Content path folder. These files contain embedded media such as the 360p.html, 480p.html, 720p.html, and 1080p.html files. Each video workload plays for one minute.

A glitch is any detectable visual or auditory defect in the user experience. The assessment provides test results for video glitches. It reports glitches for one-second intervals throughout the running of the video content. The video glitches are categorized as minor, medium, and major. The assessment also reports the placement of the glitches. Viewers are likely to notice when three or more consecutive frames are dropped. Additionally, although the audio stream is less likely to be interrupted than the video stream, a series of consecutive dropped video frames affect the viewer experience because the audio track is briefly not synchronized with the video track. For more information about the results and issues produced by this assessment, see Results for the Streaming Media Performance Assessment.

Warning

Note that when the Streaming Media Assessment is configured to stream to the same device that is playing back the video (default settings), there may be audio or video glitches that are not actionable. The ideal scenario to test streaming media is to set up a remote server described here: Set Up a Remote Server for the Streaming Media Performance Assessment.

In this topic:

  • Before You Begin

  • Workloads

  • Settings

Before You Begin

Warning

The first-run help tips in Windows 8.1 can negatively affect assessment results. To disable these, run the following command from an elevated command prompt, and reboot the computer:
reg.exe add "HKLM\Software\Policies\Microsoft\Windows\EdgeUI" /v DisableHelpSticker /t REG_DWORD /d "1" /f

Warning

Run this assessment only while the desktop is full screen. Do not run this assessment if you have another Windows Store app opened side-by-side with the desktop.

Warning

When running this assessment on Windows 8.1, make sure the Collect Analysis Trace setting is unchecked when assessing expected battery life. When checked, this option will produce an incorrect estimation.
Enable analysis trace collection only when you need additional information to investigate other energy-related problems.

To prepare to run the assessment, follow these steps:

  1. Exit any open applications.

Important

Do not touch the mouse or keyboard after the assessment job starts. Running other applications or jobs at the same time as the assessment job can affect your results. Also, the Streaming Media Performance assessment does not include any measurement of seeking or pausing functionality. Do not seek or pause the video during the assessment.

  1. Complete the appropriate steps to set up the assessment job on a single computer or on two computers that are connected over a network, as shown in the following table.
Job type Description

Run the job on a single computer

To run the job on a single computer, install the Windows® Assessment and Deployment Kit (Windows ADK) Technical Reference and open Windows® Assessment Console. Then, follow the instructions that are provided in the introduction to run the Streaming Media Performance assessment.

The job runs automatically in Internet Explorer 10 or later versions. The job runs in the Internet Explorer Media pipeline in full-screen mode. We do not recommend user interaction.

Note
By default, a 1080p, 30-fps video is available to the job. The assessment plays on the full computer screen. Additional workloads include 360p at 30 fps, 480p at 30 fps, and 720p at 30 fps.

When the job runs on a single computer, the job results include the number of video or audio glitches that occur.

Run the job on two computers

To run the job on two networked computers, install the Windows ADK Technical Reference on both computers, open Windows Assessment Console, select the Streaming Media Performance assessment, and then configure the settings as shown in Settings.

The local computer streams content from the HTTP streaming server application that is installed on the second or remote computer.

Before the assessment starts, the Streaming Media Performance assessment runs some pre-checks on the system. When these pre-checks fail, the assessment will generate errors and warnings. Although errors block the assessment from running, warnings do not block the assessment from running. You can make the adjustments that are recommended in the warning message, and continue to run the job. Unaddressed warnings and errors are reported in the results. For more information about the results and issues produced by this assessment, see Results for the Streaming Media Performance Assessment.

System Requirements

You can run this assessment only on a computer that is running Windows 8 and Internet Explorer 10.

Supported architectures include x86-based, x64-based systems, and ARM-based systems.

There are two ways to run this assessment on Windows RT:

Workloads

A workload is a set of automated tasks that simulate user activity in a predefined, repeatable manner. The workloads run independent of each other. The assessment plays video content from several workloads that range from low resolution to high resolution.

You can select any combination of these workloads to run during the assessment.

  • 360p (30 FPS) Workload

  • 480p (30 FPS) Workload

  • 720p (30 FPS) Workload

  • 1080p (30 FPS) Workload

The “p” in the video resolution indicates Progressive scan, which is used in computer monitors and digital televisions. An “i” in video resolution indicates Interlaced scan, used in standard television formats. Interlaced video content is not used in this assessment.

Note

You can also use the Streaming Media assessment as a workload in an energy-efficiency job. For more information about energy-efficiency jobs, see Energy Efficiency and Create and Run an Energy Efficiency Job.

Settings

By default, this assessment uses the recommended settings. These settings are defined by Microsoft to ensure that the results can be compared across different computer configurations or over time on the same computer. When you review the results, the run information includes metadata that indicates that the recommended settings were used so that you can easily identify results that use recommended settings in addition to those that don’t.

You can also customize these settings if you want to gather different data than what is captured by default. For example, you might identify specific data that would help you do a more detailed analysis of a particular aspect of the computer.

The following table describes the assessment settings, recommended setting values, and alternative values for each setting.

Setting Description

Use recommended settings

Specifies whether the assessment uses the recommended settings. By default, this check box is selected. To change the settings for this assessment, you must first clear this check box.

Iterations

Specifies the number of times that the assessment runs. By default, the value is 3. You will see the video play 5 times during 3 iterations. The first is a 15 second to initialize Internet Explorer, 3 video playbacks are for calculating the metrics and one is for assessing the results.

Content Path

Specifies the path of the source directory for the workload dataset that contains media and HTML files that the assessment uses. By default, the content is picked up from ../Content/Streaming Media. You can specify you own value.

Server Name

Specifies the name of the server on the local network. Even though the box appears to be blank, the path of the default server is defined. If the setting is not specified, the assessment will start the streaming server on the local computer.

Port

Specifies the port on which the server accepts requests. By default, the port on which the server accepts requests is defined as 80. You can also specify your own value.

Streaming Time

Specifies the maximum time, in seconds, that the assessment waits for a video workload to finish playing. By default, the value for this setting is 65. You can also specify your own value.

ETW Trace File Name

Specifies the name of the trace file that collects the Event Tracing for Windows (ETW) events that the assessment generates. Even though the box appears to be blank, the trace file for collecting events is defined. You can also specify your own value.

Disable ETW Tracing

Disables ETW tracing. By default, this check box is cleared. If this check box is selected, ETW tracing is disabled and the assessment job does not generate an ETW trace file.

See Also

Tasks

Set Up a Remote Server for the Streaming Media Performance Assessment
Create and Run an Energy Efficiency Job

Concepts

Results for the Streaming Media Performance Assessment
Assessments
Energy Efficiency

Other Resources

Windows Assessment Toolkit Technical Reference
Windows Assessments Console Step-by-Step Guide