Depth Camera

Microsoft Robotics

Glossary Item Box

DSS Interop Samples: Kinect Sensor

Robotics Common: Webcam Sensor

Microsoft Robotics Developer StudioSend feedback on this topic

Depth Camera

The DepthCam Sensor service provides information from a depth camera. At present the only supported depth camera is the Kinect, which means that the DepthCam Sensor service is not really generic.

The DepthCam Sensor service provides the following operations. The Replace operation is used for notifications.

GetGets the State of the DepthCamSensor service.
HttpGetGets the State via a HTTP request, usually from a web browser.
HttpQueryUsed to request images from a web browser.
ReplaceReserved. Do not send this operation request.
SubscribeSubscribe to notifications. Only Replace messages are sent as notifications.

You can Subscribe to the DepthCam Sensor and it will send frames as notifications (Replace messages). Alternatively, you can poll the service using Get.

The state of a Webcam Sensor is represented the following properties.

DepthImageshort[]This is the depth data arranged in scanline order from top to bottom and left to right as a single-dimensional array. Values are in millimeters.
FurtherThanMaxDepthValueshortThe value of a pixel when it is further than the maximum range.
MaximumRangedoubleMaximum depth value in meters.
MinimumRangedoubleMinimum depth value in meters.
NearerThanMinDepthValueshortThe value of a pixel when it is nearer than the minimum range.
PosePoseDefines the position and orientation of the camera. This must be set in an initial state (config) file.
TimestampDateTimeTime when this image was captured.
VisibleImagebyte[]The raw visible image data in BGR24 format. Note that the order is not RGB.
DepthImageSizeSizeThe dimensions of the depth image.
FieldOfViewdoubleHorizontal Field of View of the camera in Radians.
ProjectionMatrixMatrixA matrix that defines the conversion between real world view space (3D) and camera space (pixel coordinates plus depth).
InverseProjectionMatrixMatrixInverse of the Projection Matrix, i.e. it converts in the opposite direction.
Reserved for future use.

Note that the state includes both the depth data (as an array of shorts) and the RGB data.

The depth data is 11 bits and is in millimeters measured from the camera as shown in the diagram below:


The depth data includes a player index in the low-order three bits. The depth data is therefore shifted to the left by 3 bits. You must take this into account when reading the depth data.

Depth Image space consists of X and Y coordinates of pixels in the image (normalized from 0 to 1) as well as a depth (Z) value in millimeters. The pixel values do not represent anything in physical space. Note that depth pixels do not align perfectly with RGB pixels for several reasons. The Kinect service provides a way to map depth pixels to color (RGB) pixels.

Skeletons are represented as a set of joints in 3D Skeleton Space. See the Kinect Sensor for more details.

For more information, see the Kinect Services for RDS document in the Documentation folder of your RDS installation.


DSS Interop Samples: Kinect Sensor

Robotics Common: Webcam Sensor



© 2012 Microsoft Corporation. All Rights Reserved.