Camera Color Conversion (YCbCr to ARGB) for Windows Phone

The ARGB format specifies the alpha, red, green, and blue components that define color in Silverlight. When you acquire a frame from the camera preview buffer using the GetPreviewBufferArgb32(Int32[]) method, the image is returned in a one-dimensional array of integer (Int32) values. These values correspond to the 32-bit ARGB color palette that Silverlight uses to paint the UI. Each 32-bit integer in the preview buffer array corresponds to a single pixel in the image. Each of the 4 bytes within an integer corresponds to the four ARGB components as follows:

• Alpha byte (bits 25-32): Represents transparency. Frames from the camera will always be opaque, represented by a decimal value of 255.

• Red byte (bits 17-24): Represents the red component. Decimal values range from 0 to 255.

• Green byte (bits 9-16): Represents the green component. Decimal values range from 0 to 255.

• Blue byte (bits 1-8): Represents the blue component. Decimal values range from 0 to 255.

For example, a 32-bit integer that describes the color mango is illustrated in the following figure.

Although the preview buffer is returned as a one-dimensional array, the two-dimensional image can be inferred from the width and height specified by the PreviewResolution property of the PhotoCamera class. For example, if the camera buffer were only 12 pixels, GetPreviewBufferArgb32 would return an array of 12 integers to describe the 12 pixels. If the corresponding PreviewResolution was 4 x 3 and the array was applied to the Pixels property of a WriteableBitmap control, the bitmap image would be assembled by arranging the 12 pixels left-to-right and top-to-bottom, as illustrated in the following figure.

In this figure, the numbers correspond to the index value for each of the 12 integers in the array. For an example of how to use the GetPreviewBufferArgb32 method, see How to: Work with Grayscale in a Camera Application for Windows Phone.

The YCbCr format specifies the luminance and chrominance components of a frame from the camera. Luminance, Y, describes the intensity of the pixels; by itself, it is a grayscale version of the image. Chrominance, Cb and Cr, describes the color-difference components for blue and red. When you acquire a frame from the camera preview buffer using the GetPreviewBufferYCbCr(Byte[]) method, the image is returned in a one-dimensional array of byte values.

The following figure illustrates the luminance and chrominance components that are specified in the YCbCr format.

In this figure, each luminance and chrominance component is labeled with a value relative to 1 and the corresponding unsigned byte value (in parentheses). The unsigned byte values correspond to the values for Y, Cb, and Cr in the byte array that is provided by the GetPreviewBufferYCbCr method. For example, a Y value of 0.5 corresponds to a byte value of 127, a Cb value of 0.0 corresponds to a byte value of 127, and a Cr value of -1.0 corresponds to a byte value of 0.

Unlike the ARGB format, there is not a one-to-one relationship between the values in the preview buffer array and the pixels in the image. To optimize the performance of the camera, each Windows Phone device “quarter-samples” the chrominance components of the frame. This means that for every 4 bytes of Y information, only one byte of Cb and one byte of Cr is used to describe the corresponding 4 pixels of the image. Furthermore, some of the bytes in the array may be unused to allow room for padding (potentially varying between device models). Because the layout of the YCbCr preview buffer array can change, use the YCbCrPixelLayout class to reference the appropriate bytes in the preview buffer. YCbCrPixelLayout is available for each camera as a property of the PhotoCamera class.

The properties of the YCbCrPixelLayout class describe how Y, Cb, and Cr bytes are arranged in the preview buffer array. The Offset properties (YOffset, CbOffset, and CrOffset) specify the starting position for the Y, Cb, and Cr bytes in the array. The XPitch properties (YXPitch, CbXPitch, and CrXPitch) specify the number of bytes to the next Y, Cb, and Cr byte in the current row of the frame, respectively. The Pitch properties (YPitch, CbPitch, and CrPitch) specify the number of bytes between rows.

For example, the following figure of an array for a 16-pixel buffer illustrates the YCbCrPixelLayout properties in a commonly used YCbCr layout.

The YCbCrPixelLayout properties corresponding to the preview buffer illustrated in this figure have the following values.

This example also demonstrates how a 2-byte padding of the buffer would affect the layout of the Y, Cb, and Cr bytes. Although the frame in this example is only 4 pixels wide, it takes 6 bytes to reach the next row of corresponding bytes. This is reflected in the Pitch values being equal to 6. The YCbCr buffer layout can change between device models; it is important to use the YCbCrPixelLayout properties to index the appropriate luminance and chrominance components.

Using the PreviewResolution property from the corresponding PhotoCamera object, you can determine the relationship between the bytes in the preview buffer array and the pixels in the frame. Continuing this example of a 16 pixel frame, the following figure illustrates the relationship of pixels and bytes.

Important Note:
When working with YCbCr preview buffers, use the RequiredBufferSize property to determine the size of the buffer and the PreviewResolution property to determine the dimensions of the corresponding frame. YCbCr preview buffers may contain unused bytes to allow room for padding.

Luminance-Only Processing

Because the YCbCr format is more compact than ARGB, the GetPreviewBufferYCbCr(Byte[]) method provides an efficient option for image manipulation. This section describes how to select a pixel in the camera preview buffer and convert it from the YCbCr format to the ARGB format.

Tip:
The code examples in this section are excerpts from the Camera Color Picker Sample. To download the complete source code, see Code Samples for Windows Phone.

Obtaining Y, Cb, and Cr Values from the Preview Buffer

private void GetYCbCrFromPixel(YCbCrPixelLayout layout, byte[] currentPreviewBuffer, int xFramePos, int yFramePos, out byte y, out int cr, out int cb)
{
    // Find the bytes corresponding to the pixel location in the frame.
    int yBufferIndex = layout.YOffset + yFramePos * layout.YPitch + xFramePos * layout.YXPitch;
    int crBufferIndex = layout.CrOffset + (yFramePos / 2) * layout.CrPitch + (xFramePos / 2) * layout.CrXPitch;
    int cbBufferIndex = layout.CbOffset + (yFramePos / 2) * layout.CbPitch + (xFramePos / 2) * layout.CbXPitch;

    // The luminance value is always positive.
    y = currentPreviewBuffer[yBufferIndex];

    // The preview buffer contains an unsigned value between 255 and 0.
    // The buffer value is cast from a byte to an integer.
    cr = currentPreviewBuffer[crBufferIndex];
            
    // Convert to a signed value between 127 and -128.
    cr -= 128;

    // The preview buffer contains an unsigned value between 255 and 0.
    // The buffer value is cast from a byte to an integer.
    cb = currentPreviewBuffer[cbBufferIndex];

    // Convert to a signed value between 127 and -128.
    cb -= 128;
}

YCbCr to ARGB Conversion Using Integer-Only Division

private int YCbCrToArgb(byte y, int cb, int cr)
{
    // Individual RGB components.
    int r, g, b;

    // Used for building a 32-bit ARGB pixel.
    uint argbPixel;

    // Assumes Cb & Cr have been converted to signed values (ranging from 127 to -128).
            
    // Integer-only division.
    r = y + cr + (cr >> 2) + (cr >> 3) + (cr >> 5);
    g = y - ((cb >> 2) + (cb >> 4) + (cb >> 5)) - ((cr >> 1) + (cr >> 3) + (cr >> 4) + (cr >> 5));
    b = y + cb + (cb >> 1) + (cb >> 2) + (cb >> 6);

    // Clamp values to 8-bit RGB range between 0 and 255.
    r = r <= 255 ? r : 255;
    r = r >= 0 ? r : 0;
    g = g <= 255 ? g : 255;
    g = g >= 0 ? g : 0;
    b = b <= 255 ? b : 255;
    b = b >= 0 ? b : 0;

    // Pack individual components into a single pixel.
    argbPixel = 0xff000000; // Alpha
    argbPixel |= (uint)b;
    argbPixel |= (uint)(g << 8);
    argbPixel |= (uint)(r << 16);

    // Return the ARGB pixel.
    return unchecked((int)argbPixel);
}

Other Resources