Storage Performance EMMC

The Storage Performance EMMC test is an automated test that evaluates the performance of an EMMC drive.

Test details

 

Associated requirement(s)

System.Fundamentals.StorageAndBoot.BootPerformance

See the system hardware requirements.

Platforms

Windows RT (ARM-based)
Windows 8 (x86)

Run time

~4 hours

Running the test

Before you run the test, complete the test setup as described in the test requirements: System Fundamentals Testing Prerequisites.

The EMMC device must be attached to the appropriate controller. The test is non-destructive—no files or partitions will be destroyed during testing. Files will, however, be written to the drive. It is important to minimize the amount of activity occurring on the drive outside of the logo test. Since this is a performance test, outside activity may affect the results.

Troubleshooting

Check the WTT Trace by performing the following steps:

  • Go into the Child Job Results of RunJob – Storage Performance Library.

  • View Task Log of Run StorPerf.

  • Open the log file StorPerf.wtl.

  • Check for messages that may solve the issue.

Span Size Failure

  • “The requested span size, 10737418240 bytes, is larger than the precondition file, 7195066368 bytes. Logo test invalidated.”

  • If the requested span size is larger than the precondition file, an error will be logged, but the test will continue to run. The testzone.tmp file created is too small to sufficiently test the range required by the test.

  • More space must be made to create this file, or the file was too small and not properly deleted before beginning testing.

  • Currently the minimum size of the preconditioning file required is 10Gb. 10% free space must also be left on the drive. The preconditioning phase writes a file that fills all free space leaving 10% of the total free space on the drive.

    • Total Disk Size * 10% + 10GB < Free Space

Check individual test results

  • Browse Job Logs of Storage Performance EMMC / Storage Performance USB3

  • There are several types of files for each test case run inside of the test that are copied back to the controller for triage. These files contain more information than is available in the WTT logs.

  • The .result files are the console output of each process that is launched from this test.

  • The .xml files are generated by the workload that this tests launches. It is what is parsed to get the metrics.

  • The .csv file is the aggregate of all of the parsed data for every test case.

  • The naming of the .result and .xml files uniquely identifies the test case run.

    • Scen = Scenario

    • The long hex string identifies all of the parameters passed to the workload.

    • The three digit hex string is the thread id.

    • The last digits are the runs of this exact same test case on the same workload.

  • If an error occurs in the log, the first place to check is the .result file of the same name as the test case where the error occurred.

    • If it is unable to open a handle to the disk, it may not have a partition on the drive, or may not be running as administrator.

  • If there is a discrepancy about the metric, the values are located in the .xml file.

  • If there is a discrepancy about the variance or test case interactions, the .csv file shows the results for all of the test

Issues with open ETW logs

  • If the test is closed during execution, it is possible that an ETW log may remain active.

  • The easiest way to reset it is to restart the computer.

  • Manually close logger

    • Open an elevated cmd window.

      • Right click cmd.exe (C:\Windows\System32\cmd.exe) and choose Run as administrator.

    • Run logman query -ets

    • Run logman stop -ets "Circular BitLocker Logger"

If you cannot obtain a successful test result, open a support case by using your Windows Live® ID to sign in to Microsoft Support, and then visit Microsoft Support Options.

For additional troubleshooting information, see Troubleshooting the HCK.

This test returns Pass or Fail. To review test details, review the test log from Windows® Hardware Certification Kit (Windows HCK) Studio.

More information

The job takes in the device instance id of the device under test and converts the device instance id to a physical drive number or drive letter depending on the scenario. If it is required by the scenario, the job partitions and formats the drive to get it into the configuration needed for testing. The test will run through a series of test cases each mapped to items in the requirements. The test cases are self-contained and are run sequentially. A list of test cases can be obtained by using the PrintPolicy command line option with the appropriate device specified. Each of these test cases can be run on the command line using the test in standalone mode for further testing or debugging.

The Storage Performance test stores a policy table defining for each type of device what performance tests are to be run and what the appropriate metrics should be. Once the appropriate items in the table are selected, the test will sequentially spawn instances of StorageAssessment to test the items specified in the table for that device. Once StorageAssessment has finished doing its testing and created the results, Storage Performance test will parse those values and compare them with the bars defined in the logo requirements in order to print pass/fail logs.

A device is defined by three command line parameters and correspondingly used as lookups for the entries in the policy table. These three values are: Type, Protocol, and Usage. Invoking help from the command line will give all of the possible values for these items.

A scenario is defined by a Workload, Access, Operation, Operation Value, IO Size, Span Size, Runtime, and Precondition. Each scenario defined in the table will correspond to the workload being spawned once. Multiple of the same scenarios defined for one device will still only invoke one test case.

A metric is defined by its type and value. Intrinsic to the metric is its units and whether the bar is an upper limit or lower limit. Many metrics can be specified for each scenario which results in only one test case being invoked for that scenario. Also defined for each metric is a Boolean value specifying if the metric should log an assessment score.

For each entry in the table there is a variance criteria specified which defines the maximum variance allowed over the last set of runs before it will stop testing that test case. For many of the entries it is defined as a minimum of 5 runs, maximum of 30 runs and the variance over the last 5 runs must be under 10% to continue testing. The test case will be rerun up to 30 times or until the variance requirement is satisfied. At that point, the metric will be evaluated according to the defined properties of the metric (minimum, maximum, mean, average, and so on) over the last set of runs.

Although Storage Performance test is not limited to one workload, the majority of the scenarios defined in the policy table use the StorageAssessment workload in order to generate the performance workload and metrics. StorageAssessment is based on WinSat, but contains many other features allowing it to be used in Logo testing. Storage Assessment uses ETW tracing in order to track and assess the disk with the IO patterns it generates.

Command syntax

 

Command option Description

DriveNumber <number>

Physical drive number of device under test

DriveLetter <letter>

Drive letter of device under test

DeviceType <usb2 | usb3 | emmc | usb3_pw_ssd | usb3_pw_rt>

Device properties.

Protocol <bot | uas>

Device properties.

Usage <boot | data>

Device properties.

Workload <diskspd | storageassessment | diskio>

Workload type. Invalidates logo test.

IOAccess <sequential | random>

Access type. Invalidates logo test.

IOOperation <read | write | flush>

Operation type. Invalidates logo test.

IOOperationValue <value>

Operation value. Invalidates logo test.

IOSize <size>

IO size. Invalidates logo test.

Span <size>

Span size. Invalidates logo test.

Runtime <duration>

Duration in seconds to run the test case. Invalidates logo test.

Precondition <true | false>

Preconditioning. Invalidates logo test.

QueueDepth <depth>

Queue depth. Invalidates logo test.

Cooldown <duration>

Duration in seconds to wait after each test case. Invalidates logo test.

PrintPolicy

Prints the policy table.

File list

 

File Location

StorPerf.exe

<[testbinroot]>\NTTest\driverstest\storage\wdk\

StorageAssessment.exe

<[testbinroot]>\NTTest\driverstest \storage\wdk\StorageAssessment

ssdtest.dat

<[testbinroot]>\NTTest\driverstest \storage\wdk\StorageAssessment

Show:
© 2015 Microsoft