Export (0) Print
Expand All

Kinect Sensor

Microsoft Robotics

Glossary Item Box

Robotics Common: Depth CameraWebcam Sensor

Microsoft Robotics Developer StudioSend feedback on this topic

Kinect Sensor

Hh438998.Kinect(en-us,MSDN.10).png

The Kinect Services support the following features:

  • Depth image including Player Index
  • RGB image
  • Tilt (Get and Set)
  • Microphone Array (not in simulation)
  • Skeleton Tracking (not in simulation)

You can specify the resolution of the Depth and RGB cameras independently via a config file, as well as the depth camera mode.The config file also specifies whether you want skeleton tracking to be performed or not. If you do not use the skeleton data, you should not track it because there is a performance overhead. You cannot turn skeleton tracking on once the service is running, so it must be selected in the config file.

The Kinect depth sensor range is: minimum 800mm and maximum 4000mm. The Kinect for Windows Hardware can however be switched to Near Mode which provides a range of 500mm to 3000mm instead of the Default range. If you are using an Xbox Kinect with the Kinect for Windows SDK then Near Mode is not supported.

The Kinect uses Infrared so it can see through glass. Therefore it cannot be used reliably for obstacle avoidance if you have glass doors. Also because it uses IR, the Kinect will not work in direct sunlight, e.g. outdoors.

The Kinect service provides the following operations.

OperationDescription
DepthImageToSkeletonConverts a pixel from Depth Image coordinates to Skeleton coordinates.
DepthToColorImageConverts a pixel from Depth coordinates to Color Image coordinates.
GetGets the State of the Kinect service (listed above).
GetDepthPropertiesGets the values of all the depth properties: Maximum, Minimum, TooFar, TooNear and Unknown.
HttpGetGets the State via a HTTP request, usually from a web browser.
HttpPostUpdates the State via a HTTP request. Used by the XSLT page to make changes.
HttpQueryUsed to request different images from a web browser.
QueryRawFrameUsed to get raw data from the Kinect. You can request any combination of Video, Depth and Skeleton frames.
SetFrameRateSets the desired Frame Rate.
SkeletonToColorImageConverts a pixel from Skeleton coordinates to Color Image coordinates.
SkeletonToDepthImageConverts a pixel from Skeleton coordinates to Depth Image coordinates.
SubscribeSubscribe to notifications. Only SetFrameRate and UpdateTilt generate notifications.
UpdateSkeletalSmoothingUsed to update the skeletal smoothing parameters and turn smoothing on or off.
UpdateTiltAdjusts the tilt angle of the camera.

You can issue QueryRawFrame requests to the Kinect service and ask for any combination of depth, RGB and skeleton joint data. You cannot obtain this information via notifications.

Use the UpdateTilt operation to tilt the camera up or down. If you want to know the current tilt angle, you can do a Get and read it from the state. Note that the actual angle depends on whether the whole Kinect sensor is tilted or not. It has an accelerometer that allows it to sense the direction of gravity, i.e. which way is down.

The state of a Kinect is represented the following properties.

NameTypeDescription
ColorImageFormatColorImageFormat*The format of the Color (Video) image (as an enum value).
DepthImageFormatDepthImageFormat*The format of Depth image (as an enum value).
DepthStreamRangeDepthRange*The range to use for Depth data (as an enum value - Default or Near). The Near value can only be used with
DeviceIDintThe ID of the selected Kinect camera.
FrameRateDoubleThe target frame rate in frames per second (fps). This cannot exceed MaxFrameRate.
IsDepthServiceUpdateEnabledboolEnables notifications on depth images. Turn it off to improve performance and then query the Kinect for data.
IsWebCamServiceUpdateEnabledboolEnables notifications on the video images. Turn it off to improve performance and then query the Kinect for data.
MaxFrameRatedoubleThe maximum frame rate that can be set (in fps). This is a constant and cannot be changed.
SkeletalEngineTransformSmoothParametersTransformSmoothParameters*The parameters for skeletal data smoothing.
TiltDegreesdoubleThe current camera tilt angle in degrees. The valid range is -27 to +27.
TransformSmoothBoolEnables skeletal data smoothing.
UseColorboolUsed to enable the color video stream at startup. See the documentation for the Kinect for Windows SDK.
UseDepthboolUsed to enable the Depth stream at startup.
UseSkeletalTrackingboolUsed to enable Skeleton Tracking at startup.

Data types marked with an asterisk (*) come from the Kinect for Windows SDK. Refer to the Kinect for Windows SDK documentation for more information.

Note that the state can be set using a config file. Many of the values cannot be modified after the service has been started.

The skeleton data consists of a set of joints. These joints are shown in the diagram below.

Hh438998.SkeletonJoints(en-us,MSDN.10).png

The coordinate system for the skeleton data is a full 3D system with values in meters. It is shown in the following diagram.

Hh438998.SkeletonSpace(en-us,MSDN.10).png

There are operations for converting between Skeleton Space and Depth Image space. The Kinect for Windows SDK supports up to two players (skeletons) being tracked at the same time. A player index is inserted into the lower 3 bits of the depth data so that you can tell which depth pixels belong to which player. You must take these 3 bits into account when you want to examine the depth values.

For more information about the Kinect, see the Kinect Services for RDS document in the Documentation folder of your RDS installation. Also there are topics describing the Generic Depth Camera and Webcam Sensor contracts which are implemented for the Kinect.

Robotics Common: Depth CameraWebcam Sensor

 

 

© 2012 Microsoft Corporation. All Rights Reserved.

Show:
© 2014 Microsoft