2.2.6 Cropping Info SEI Message

The cropping info is a structure that describes the frame coordinates of one or multiple cropping windows. Cropping windows are regions of interest that can be decided by a face tracker or by senders’ preference at runtime. The provision of cropping info provides a reliable way for the receiver to properly crop a window out of the original frame without loss of important information when necessary.

This protocol defines a User Data Unregistered SEI message as the cropping info message.

The syntax of the User Data Unregistered SEI message is defined in [ISO/IEC 14496-10:2014] Annex D.

The cropping info SEI message can be embedded in a PACSI NAL unit. The PACSI NAL containing the cropping info SEI message MUST be present in the base layer and SHOULD be followed by any VCL NAL unit.

The format of cropping info SEI message is defined as follows:

F bit (1 bit): See the definition in [RFC6190] section 1.1.3.

NRI (2 bits): See the definition in [RFC6190] section 1.1.3.

Type (5 bits): See the definition in [RFC6190] section 1.1.3.

payloadType (1 byte): MUST be 5. See the definition in [ISO/IEC 14496-10:2014].

payloadSize (18 + 9 * N bytes): N is the value of the numOfCropData field. See the definition in [ISO/IEC 14496-10:2014].

uuid_iso_iec_11578 (16 bytes): [Guid("BB7FC1A0-6986-4052-90F0-0929217539CF")]

numOfCropData (1 byte): Total number of cropping info set in the message.

crop_info_type (1 byte): MUST be zero.

frame_crop_confidence_level1 (1 byte): Specify the confidence level, quantified with a value between 0 and 100 with higher value meaning higher confidence, of the first cropping window. The cropping info confidence level is decided by the crop info processor. The value zero indicates the cropping confidence is indeterminate.

frame_crop_right_offset1 (2 bytes): Specify the offset from the right edge of the first cropping window to the right edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_left_offset1 (2 bytes): Specify the offset from the left edge of the first cropping window to the left edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_top_offset1 (2 bytes): Specify the offset from the top edge of the first cropping window to the top edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_bottom_offset1 (2 bytes): Specify the offset from the bottom edge of the first cropping window to the bottom edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_confidence_level2 (1byte): Specify the confidence level, quantified with a value between 0 and 100 with higher value meaning higher confidence, of the second cropping window (if present). The cropping info confidence level is decided by the crop info processor. The value zero indicates the cropping confidence is indeterminate.

frame_crop_right_offset2 (2 bytes): Specify the offset from the right edge of the second cropping window (if present) to the right edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_left_offset2 (2 bytes): Specify the offset from the left edge of the second cropping window (if present) to the left edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_top_offset2 (2 bytes): Specify the offset from the top edge of the second cropping window (if present) to the top edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_bottom_offset2 (2 bytes): Specify the offset from the bottom edge of the second cropping window (if present) to the bottom edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_confidence_levelN (1 byte): Specify the confidence level, quantified with a value between 0 and 100 with higher value meaning higher confidence, of the nth cropping window (if present). The cropping info confidence level is decided by the crop info processor. The value zero indicates the cropping confidence is indeterminate.

frame_crop_right_offsetN (2 bytes): Specify the offset from the right edge of the nth cropping window (if present) to the right edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_left_offsetN (2 bytes): Specify the offset from the left edge of the nth cropping window (if present) to the left edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_top_offsetN (2 bytes): Specify the offset from the top edge of the nth cropping window (if present) to the top edge of the rectangular region of the coded video sequence, in terms of pixels.

frame_crop_bottom_offsetN (2 bytes): Specify the offset from the bottom edge of the nth cropping window (if present) to the bottom edge of the rectangular region of the coded video sequence, in terms of pixels.

The following diagram is an example of the offsets defined earlier in this section.

Figure 1: Offsets example