Share via


Kinect Sensor

Glossary Item Box

Robotics Common: Depth CameraWebcam Sensor

See Also Microsoft Robotics Developer Studio Send feedback on this topic

Kinect Sensor

Hh438998.Kinect(en-us,MSDN.10).png

The Kinect Services support the following features:

  • Depth image including Player Index
  • RGB image
  • Tilt (Get and Set)
  • Microphone Array (not in simulation)
  • Skeleton Tracking (not in simulation)

You can specify the resolution of the Depth and RGB cameras independently via a config file, as well as the depth camera mode.The config file also specifies whether you want skeleton tracking to be performed or not. If you do not use the skeleton data, you should not track it because there is a performance overhead. You cannot turn skeleton tracking on once the service is running, so it must be selected in the config file.

The Kinect depth sensor range is: minimum 800mm and maximum 4000mm. The Kinect for Windows Hardware can however be switched to Near Mode which provides a range of 500mm to 3000mm instead of the Default range. If you are using an Xbox Kinect with the Kinect for Windows SDK then Near Mode is not supported.

The Kinect uses Infrared so it can see through glass. Therefore it cannot be used reliably for obstacle avoidance if you have glass doors. Also because it uses IR, the Kinect will not work in direct sunlight, e.g. outdoors.

The Kinect service provides the following operations.

Operation Description
DepthImageToSkeleton Converts a pixel from Depth Image coordinates to Skeleton coordinates.
DepthToColorImage Converts a pixel from Depth coordinates to Color Image coordinates.
Get Gets the State of the Kinect service (listed above).
GetDepthProperties Gets the values of all the depth properties: Maximum, Minimum, TooFar, TooNear and Unknown.
HttpGet Gets the State via a HTTP request, usually from a web browser.
HttpPost Updates the State via a HTTP request. Used by the XSLT page to make changes.
HttpQuery Used to request different images from a web browser.
QueryRawFrame Used to get raw data from the Kinect. You can request any combination of Video, Depth and Skeleton frames.
SetFrameRate Sets the desired Frame Rate.
SkeletonToColorImage Converts a pixel from Skeleton coordinates to Color Image coordinates.
SkeletonToDepthImage Converts a pixel from Skeleton coordinates to Depth Image coordinates.
Subscribe Subscribe to notifications. Only SetFrameRate and UpdateTilt generate notifications.
UpdateSkeletalSmoothing Used to update the skeletal smoothing parameters and turn smoothing on or off.
UpdateTilt Adjusts the tilt angle of the camera.

You can issue QueryRawFrame requests to the Kinect service and ask for any combination of depth, RGB and skeleton joint data. You cannot obtain this information via notifications.

Use the UpdateTilt operation to tilt the camera up or down. If you want to know the current tilt angle, you can do a Get and read it from the state. Note that the actual angle depends on whether the whole Kinect sensor is tilted or not. It has an accelerometer that allows it to sense the direction of gravity, i.e. which way is down.

The state of a Kinect is represented the following properties.

Name Type Description
ColorImageFormat ColorImageFormat* The format of the Color (Video) image (as an enum value).
DepthImageFormat DepthImageFormat* The format of Depth image (as an enum value).
DepthStreamRange DepthRange* The range to use for Depth data (as an enum value - Default or Near). The Near value can only be used with
DeviceID int The ID of the selected Kinect camera.
FrameRate Double The target frame rate in frames per second (fps). This cannot exceed MaxFrameRate.
IsDepthServiceUpdateEnabled bool Enables notifications on depth images. Turn it off to improve performance and then query the Kinect for data.
IsWebCamServiceUpdateEnabled bool Enables notifications on the video images. Turn it off to improve performance and then query the Kinect for data.
MaxFrameRate double The maximum frame rate that can be set (in fps). This is a constant and cannot be changed.
SkeletalEngineTransformSmoothParameters TransformSmoothParameters* The parameters for skeletal data smoothing.
TiltDegrees double The current camera tilt angle in degrees. The valid range is -27 to +27.
TransformSmooth Bool Enables skeletal data smoothing.
UseColor bool Used to enable the color video stream at startup. See the documentation for the Kinect for Windows SDK.
UseDepth bool Used to enable the Depth stream at startup.
UseSkeletalTracking bool Used to enable Skeleton Tracking at startup.

Data types marked with an asterisk (*) come from the Kinect for Windows SDK. Refer to the Kinect for Windows SDK documentation for more information.

Note that the state can be set using a config file. Many of the values cannot be modified after the service has been started.

The skeleton data consists of a set of joints. These joints are shown in the diagram below.

Hh438998.SkeletonJoints(en-us,MSDN.10).png

The coordinate system for the skeleton data is a full 3D system with values in meters. It is shown in the following diagram.

Hh438998.SkeletonSpace(en-us,MSDN.10).png

There are operations for converting between Skeleton Space and Depth Image space. The Kinect for Windows SDK supports up to two players (skeletons) being tracked at the same time. A player index is inserted into the lower 3 bits of the depth data so that you can tell which depth pixels belong to which player. You must take these 3 bits into account when you want to examine the depth values.

For more information about the Kinect, see the Kinect Services for RDS document in the Documentation folder of your RDS installation. Also there are topics describing the Generic Depth Camera and Webcam Sensor contracts which are implemented for the Kinect.

See Also 

Robotics Common: Depth CameraWebcam Sensor

 

 

© 2012 Microsoft Corporation. All Rights Reserved.