Video Interlacing

This topic describes how media sources and decoders should handle interlaced video content.

To decode and render interlaced video correctly, the following information is needed:

  • Progressive or interlaced. A video stream can contain progressive frames, interlaced frames, or a mix of both.

  • Field dominance. Field dominance describes which field appears first, the upper field or the lower field.

  • Repeat first field. This flag is used in 3:2 pulldown, when the frame is progressive but the stream is interlaced. In this context, the first field can be the upper or lower field.

  • Interleaved fields or single field. A sample can hold either a single field, or two interleaved fields. If a sample contains a single field, the sample height is half the frame height, because the sample contains only half of the scan lines for a frame. Interleaved fields are recommended unless the characteristics of the source content dictate otherwise.

Any of these characteristics can change from one sample to the next. However, video components need to know something about the overall content before streaming begins. For example, if the video is interlaced, the enhanced video renderer (EVR) needs to reserve video memory for the deinterlacing. If the video is entirely progressive frames, on the other hand, the EVR can optimize the rendering pipeline. Adding a deinterlacing step to the pipeline increases the rendering latency.

Information about interlacing is stored in two places:

  • General information about the interlacing in a stream is placed in the media type. For more information about media types, see Media Types.

  • Information that can change with each sample is placed on the sample as an attribute. For more information about samples, see Media Samples.

Interlace Information in the Media Type

The MF_MT_INTERLACE_MODE attribute on the media type describes how the stream as a whole is interlaced. The value of this attribute is a member of the MFVideoInterlaceMode enumeration. A video media type should always have this attribute.

  • If the stream contains only progressive frames, with no interlaced frames, use MFVideoInterlace_Progressive.
  • If the stream contains only interlaced frames, and every sample contains two interleaved fields, use MFVideoInterlace_FieldInterleavedUpperFirst or MFVideoInterlace_FieldInterleavedLowerFirst.
  • If the stream contains only interlaced frames, and every sample contains a single field, use MFVideoInterlace_FieldSingleUpper or MFVideoInterlace_FieldSingleLower. If the fields alternate between upper and lower, then it does not matter which of these two values is used. If the format contains just upper fields, or just lower fields, then set the value that corresponds to the content.
  • If the stream contains a mix of interlaced and progressive frames, or if the field dominance switches, set the media type to MFVideoInterlace_MixedInterlaceOrProgressive. Use sample attributes to describe each frame.

The following table summarizes this attribute.

MF_MT_INTERLACE_MODEInterlaced?SamplesFirst field
MFVideoInterlace_ProgressiveNoProgressive frameNot applicable
MFVideoInterlace_FieldInterleavedUpperFirstYesInterleaved fieldsUpper first
MFVideoInterlace_FieldInterleavedLowerFirstYesInterleaved fieldsLower first
MFVideoInterlace_FieldSingleUpperYesSingle fieldUpper first
MFVideoInterlace_FieldSingleLowerYesSingle fieldLower first
MFVideoInterlace_MixedInterlaceOrProgressiveCan varyInterleaved fields or progressive framesCan vary

 

Interleaved fields and single fields cannot be mixed. Switching from one to another requires a media type change.

Interlace Flags on Samples

Information that can change from one sample to the next is indicated using sample attributes. Use the IMFSample interface to get or set these attributes.

All of the interlacing attributes listed in this section have Boolean values. Effectively, each of these attributes can have three values: either TRUE, FALSE, or not set. If an attribute is not set, the value is taken from the media type. If an attribute is set, the value overrides the media type. Some combinations of flags and media types are not valid.

AttributeDescription
MFSampleExtension_Interlaced

If TRUE, the frame is interlaced. If FALSE, the frame is progressive.

Set this attribute on every sample if the media type is MFVideoInterlace_MixedInterlaceOrProgressive.

MFSampleExtension_BottomFieldFirst

The meaning of this flag depends on whether the samples contain interleaved fields or single fields.

  • Interleaved fields: If TRUE, the lower field is first. If FALSE, the upper field is first.

  • Single fields: If TRUE, the sample contains a lower field. If FALSE, the sample contains an upper field.

Set this attribute on every sample if the media type is MFVideoInterlace_FieldSingleUpper, MFVideoInterlace_FieldSingleLower, or MFVideoInterlace_MixedInterlaceOrProgressive.

MFSampleExtension_RepeatFirstField If TRUE, the first field is repeated. If FALSE or not set, the first field is not repeated.
MFSampleExtension_SingleField If TRUE, the sample contains a single field. If FALSE, the sample contains interleaved fields.

 

The following table shows which flags are required, optional, or prohibited, based on the media type.

Media TypeInterlaced FlagBottomFieldFirst FlagRepeatFirstField FlagSingleField Flag
ProgressiveOptional; if set, must be FALSE.Do not set.Do not set.Do not set.
Interleaved fieldsOptional; if set, must be TRUE.Optional; if set, must match media type.Do not set.Optional; if set, must be FALSE.
Single fieldsOptional; if set, must be TRUE.Required.Do not set.Set to TRUE.
MixedRequired.Required.Required.Optional; if set, must be FALSE.

 

In the cases where the attribute is optional, the media type already defines the information. It is valid to set the attribute to match, but not required.

For example, if the media type is MFVideoInterlace_Progressive, it implies that all frames in the stream are progressive. Therefore, you can either set the MFSampleExtension_Interlaced attribute to FALSE, or leave the attribute unset.

Recommendations

This section contains recommendations for various types of content.

1. The video is all progressive frames.

  • Set the media type to MFVideoInterlace_Progressive.

  • Do not set the MFSampleExtension_Interlaced attribute, or set it to FALSE on every frame.

  • Do not set the MFSampleExtension_BottomFieldFirst, MFSampleExtension_RepeatFirstField, or MFSampleExtension_SingleField attributes.

2. The video is all interlaced fields with the same field dominance. Samples contain interleaved fields.

  • Set the media type to MFVideoInterlace_FieldInterleavedUpperFirst or MFVideoInterlace_FieldInterleavedLowerFirst.

  • Do not set the MFSampleExtension_Interlaced attribute, or set it to TRUE on every frame.

  • Do not set the MFSampleExtension_BottomFieldFirst attribute, or set the value on every frame to match the media type.

  • Do not set the MFSampleExtension_RepeatFirstField attribute, or set it to FALSE on every frame.

  • Do not set the MFSampleExtension_SingleField attribute, or set it to FALSE on every frame.

3. The video contains a mix of interlaced and progressive frames, with repeated fields and varying field dominance (for example, DVD video).

  • Set the media type to MFVideoInterlace_MixedInterlaceOrProgressive.

  • On every frame, set the MFSampleExtension_Interlaced, MFSampleExtension_BottomFieldFirst, and MFSampleExtension_RepeatFirstField attributes.

  • Do not set the MFSampleExtension_SingleField attribute, or set it to FALSE on every frame.

4. The video is interlaced and samples contain single fields.

  • Set the media type to MFVideoInterlace_FieldSingleUpper or MFVideoInterlace_FieldSingleLower.

  • On every frame, set the MFSampleExtension_BottomFieldFirst attribute.

  • Do not set the MFSampleExtension_Interlaced attribute, or set it to TRUE on every frame.

  • Do not set the MFSampleExtension_RepeatFirstField attribute, or set it to FALSE on every frame.

  • Do not set the MFSampleExtension_SingleField attribute, or set it to TRUE on every frame.

Most video content falls into one of these categories.

MPEG-2 Mappings

For MPEG-2 content, use the following mappings to convert the MPEG-2 flags to Media Foundation sample attributes.

picture_structure

ValueSample Attribute
frameMFSampleExtension_SingleField = FALSE
top_field

MFSampleExtension_SingleField = TRUE

MFSampleExtension_BottomFieldFirst = FALSE

bottom_field

MFSampleExtension_SingleField = TRUE

MFSampleExtension_BottomFieldFirst = TRUE

 

progressive_frame

ValueSample Attribute
0MFSampleExtension_Interlaced = TRUE
1MFSampleExtension_Interlaced = FALSE

 

top_field_first

ValueSample Attribute
0MFSampleExtension_BottomFieldFirst = TRUE
1MFSampleExtension_BottomFieldFirst = FALSE

 

repeat_first_field

ValueSample Attribute
0MFSampleExtension_RepeatFirstField = FALSE
1MFSampleExtension_RepeatFirstField = TRUE

 

Single-Field Samples

If the media type is MFVideoInterlace_FieldSingleUpper or MFVideoInterlace_FieldSingleLower, it means that each sample contains a single field. However, the media type describes the entire frame. Therefore, each buffer contains only half the number of field lines given in the media type. For example, if the media type describes the video as 720 × 480, each field contains 240 scan lines, and therefore each buffer contains only 240 rows of pixels. If you write a component that accepts media types with single-field samples, you must take this fact into account when you access the data in the buffer.

The same rule applies to the geometric aperture (MF_MT_GEOMETRIC_APERTURE attribute) and minimum display aperture (MF_MT_MINIMUM_DISPLAY_APERTURE attribute). These regions are specified in terms of the entire frame, not the individual fields.

DirectShow Mappings

In DirectShow, per-sample interlacing information is contained in the dwTypeSpecificFlags member of the AM_SAMPLE2_PROPERTIES structure. The following table shows the equivalent attributes for Media Foundation.

DirectShow sample flagMedia Foundation sample attribute
AM_VIDEO_FLAG_INTERLEAVED_FRAMEMFSampleExtension_SingleField = FALSE.
AM_VIDEO_FLAG_FIELD1

MFSampleExtension_Interlaced = TRUE.

MFSampleExtension_SingleField = TRUE.

MFSampleExtension_BottomFieldFirst = FALSE.

AM_VIDEO_FLAG_FIELD2

MFSampleExtension_Interlaced = TRUE.

MFSampleExtension_SingleField = TRUE.

MFSampleExtension_BottomFieldFirst = TRUE.

AM_VIDEO_FLAG_WEAVEMFSampleExtension_Interlaced = FALSE. (This flag indicates that the driver should not deinterlace the two fields.)
AM_VIDEO_FLAG_FIELD1FIRSTMFSampleExtension_BottomFieldFirst = FALSE. If the content is interlaced and the AM_VIDEO_FLAG_FIELD1FIRST flag is not present, set this attribute to TRUE.
AM_VIDEO_FLAG_REPEAT_FIELDMFSampleExtension_RepeatFirstField = TRUE. If the AM_VIDEO_FLAG_REPEAT_FIELD flag is not present, set this attribute to FALSE.

 

If the DirectShow sample does not contain sample flags, use the value of dwInterlaceFlags from the VIDEOINFOHEADER2 structure:

DirectShow interlace flagMedia Foundation sample attribute
AMINTERLACE_IsInterlacedMFSampleExtension_Interlaced = TRUE.
AMINTERLACE_1FieldPerSampleMFSampleExtension_SingleField = TRUE.
AMINTERLACE_Field1FirstMFSampleExtension_BottomFieldFirst = FALSE.

 

Related topics

Video Media Types
Media Types

 

 

Community Additions

ADD
Show:
© 2014 Microsoft