Reporting Miracast encode chunks and statistics

Display hardware can process each video frame sent over a Miracast wireless display link by splitting the frame into multiple parts, or encode chunks. Each chunk has a unique chunk ID that’s generated from the frame number and the frame part (or slice) number. Each chunk that’s related to the same desktop frame update must be assigned the same frame number.

Reporting chunk processing

A driver can encode a frame to be sent over a Miracast wireless link either in multiple processing steps—for example separating color conversion from encoding—or in a single step. Each processing step should be assigned a unique frame part number within the frame.

Either the Miracast user-mode driver or the display miniport driver must notify the operating system each time that the hardware has completed a processing step for a frame, and immediately before each part of the frame is sent to the network. The time of a particular reported processing step is assumed to be the time the event was reported to the operating system, so it's important to report the stages as rapidly as possible.

The operating system takes no action other than to log these events using the Event Tracing for Windows (ETW) kernel-level tracing facility. This info is nevertheless important for measuring and investigating performance issues.

These are the possible ways that a driver can provide the notification:

  • The Miracast user-mode driver calls the ReportStatistic callback function to report details with the MIRACAST_STATISTIC_TYPE_CHUNK_PROCESSING_COMPLETE type, or with MIRACAST_STATISTIC_TYPE_CHUNK_SENT to indicate the chunk is just about to be sent to the network stack for transmission.
  • The display miniport driver reports details of the chunk processing with the DXGK_INTERRUPT_MICACAST_CHUNK_PROCESSING_COMPLETE interrupt type, although this report can only be made at interrupt time, and in a addition to logging the chunk info, a chunk packet is created and queued so that the Miracast user-mode driver can retrieve it by calling the GetNextChunkData callback function.
  • The display miniport driver calls the DxgkCbReportChunkInfo callback function at any IRQL level. This function logs only the chunk info and does not queue any chunk packets.

If the desktop image is not updated but the driver needs to encode the desktop image again to improve quality, the same frame number and part numbers should be used. The performance tools will trigger the second encode complete event for the same frame and part number, indicating that a second encode of the same frame was performed.

Note  The last slice of each frame must have a frame part number of zero. Doing this indicates to performance tools that this is the last slice of the frame.

To ensure correct synchronization of the primary surface, if the encoding is performed by the pixel pipeline, any requested flip operation at a VSync interval should not be reported before the encoding has finished accessing the primary surface. This prevents the presenter from rendering to the primary surface while the encode engine is reading from it.

The Miracast user-mode driver should inform the operating system at each of several stages of processing the frame:

Start frame, chunk type MIRACAST_CHUNK_TYPE_FRAME_START:

Represents the point where the operating system asks the driver to display the new desktop frame. Although technically this could be reported from the Miracast user-mode driver, the start of processing new frame always involves the display miniport driver and should be reported by that driver.

Color convert complete, chunk type MIRACAST_CHUNK_TYPE_COLOR_CONVERT_COMPLETE:

Some solutions have separate color convert and encode stages. In such solutions the color convert complete processing event should be reported as soon as possible, and the driver should use the DXGK_MIRACAST_CHUNK_INFO.ProcessingTime member to report the time it took for the hardware to perform the operation. If the entire frame is color converted all at once rather than in slices, then the part number should be zero.

Encode complete, chunk type MIRACAST_CHUNK_TYPE_ENCODE_COMPLETE:

Indicates that the H.264 encode has been completed. The ProcessingTime and EncodeRate members of the DXGK_MIRACAST_CHUNK_INFO structure should be completed.

Frame send, calling ReportStatistic using MIRACAST_STATISTIC_TYPE_CHUNK_SENT:

Indicates that the data packet for this frame/part number is just about to be sent to the networking API for transmission from the Miracast user-mode driver. If the data for this frame/part is sent using multiple calls to the networking API, then this should only be logged just before the first packet is sent. The call to indicate this should be made just before calling the network API. This is important because if the network API blocks calls, then we do not want that blocked time to count against processing of the frame in the graphics stack.

Dropped frame, chunk type MIRACAST_CHUNK_TYPE_FRAME_DROPPED:

If at any time the driver decides that it won't complete the processing of the frame/part and send it to the sink, then it should report the dropped frame. In this context a frame is only considered dropped if the driver actually started processing it by logging MIRACAST_CHUNK_TYPE_FRAME_START. If a driver calculates that it's going to skip this frame without any processing, it can log MIRACAST_CHUNK_TYPE_FRAME_DROPPED without logging MIRACAST_CHUNK_TYPE_FRAME_START.

Driver defined chunk type MIRACAST_CHUNK_TYPE_ENCODE_DRIVER_DEFINED_1 or _2:

These chunk types are available to help you understand the performance of a scenario. An example would be where the driver uses these types to indicate that an I-Frame was created for this frame. Another example would be where the driver logs an additional packet after the last slice of the frame has been sent to network APIs that contained the total size of the encoded frame.

Here are examples of how frame color is converted and then how the display miniport driver reports completion of the color conversion.

Reporting a single frame without using slices:

Value of MIRACAST_CHUNK_INFO member:
ChunkType valueChunkId.ChunkId.
Stage of processingMIRACAST_ CHUNK_TYPE_FrameNumberPartNumberProcessingTimeEncodeRate
Start processing frameFRAME_START101000
Color conversion is completeCOLOR_CONVERT_ COMPLETE10109500
Encode is completeENCODE_ COMPLETE1010104215000
Just before call to send data to network ReportStatistic call*101, value of ChunkSent. ChunkId. FrameNumber0, value of ChunkSent. ChunkId. PartNumberN/AN/A

 

*Called using MIRACAST_STATISTIC_TYPE_CHUNK_SENT.

Reporting a single frame, processed using slices:

Value of MIRACAST_CHUNK_INFO member:
ChunkTypeChunkId.ChunkId.
Stage of processingMIRACAST_ CHUNK_TYPE_FrameNumberPartNumberProcessingTimeEncodeRate
Start processing frameFRAME_START101000
Color conversion is completeCOLOR_CONVERT_ COMPLETE10109500
Encode of slice 1 is completeENCODE_ COMPLETE1011104215000
Encode of slice 2 is completeENCODE_ COMPLETE101040015000
Just before call to send slice 1 data to network ReportStatistic call*101, value of ChunkSent. ChunkId. FrameNumber1, value of ChunkSent. ChunkId. PartNumberN/AN/A
Just before call to send slice 2 data to network ReportStatistic call*101, value of ChunkSent. ChunkId. FrameNumber0, value of ChunkSent. ChunkId. PartNumber (See Note above.)N/AN/A

 

*Called using MIRACAST_STATISTIC_TYPE_CHUNK_SENT.

Reporting an original frame, processed and then re-encoded without using slices:

Value of MIRACAST_CHUNK_INFO member:
ChunkTypeChunkId.ChunkId.
Stage of processingMIRACAST_ CHUNK_TYPE_FrameNumberPartNumberProcessingTimeEncodeRate
Start processing frameFRAME_START101000
Color conversion is completeCOLOR_CONVERT_ COMPLETE10109500
Encode is completeENCODE_ COMPLETE1010104215000
Just before call to send data for original frame to network ReportStatistic call*101, value of ChunkSent. ChunkId. FrameNumber0, value of ChunkSent. ChunkId. PartNumberN/AN/A
Re-encode is completeENCODE_ COMPLETE101050015000
Just before call to send data for re-encoded frame to network ReportStatistic call*101, value of ChunkSent. ChunkId. FrameNumber0, value of ChunkSent. ChunkId. PartNumberN/AN/A

 

*Called using MIRACAST_STATISTIC_TYPE_CHUNK_SENT.

Reporting protocol events

When the Miracast user-mode driver reports protocol events by calling the ReportStatistic function with MIRACAST_STATISTIC_DATA.StatisticType set to MIRACAST_STATISTIC_TYPE_EVENT, the operating system logs the event but takes no other action. These events are nevertheless valuable for diagnostics and performance investigation.

The MIRACAST_PROTOCOL_EVENT enumeration includes possible protocol event types that can be reported.

Reporting protocol errors

While a Miracast connected session is in progress, if a Miracast user-mode driver discovers an error, it should call the ReportSessionStatus callback function with appropriate MIRACAST_STATUS error status info in the MiracastStatus parameter. The operating session will always destroy the session when an error is reported.

Note that although the operating system merely logs the ReportSessionStatusStatus parameter for diagnostics and doesn't take any action based on its value. However, we recommend that the driver use this parameter to differentiate between different causes of the error.

 

 

Send comments about this topic to Microsoft

Show:
© 2014 Microsoft