Microsoft Kinect for Windows SDK and Toolkit – v1.5 Release Notes
Welcome to the Microsoft Kinect for Windows SDK. The Microsoft Kinect for Windows SDK provides the native and managed APIs and the tools you need to develop Kinect for Windows Applications. Developers can create rich experiences by using Microsoft Kinect for Windows sensor technology on computers running Windows 7 and Windows Embedded Standard 7. Developing Kinect for Windows Applications is essentially the same as developing other Windows applications, except that this SDK provides support for the features of the Microsoft Kinect for Windows sensor (color images, depth images, audio, skeletal data, etc.).
- License Agreement
- System Requirements
- Getting Help
- Changes in Developer Toolkit 1.5.1
- Changes in SDK 1.5/Developer Toolkit 1.5.0
- Known Issues for SDK 1.5/Developer Toolkit 1.5.1
By downloading, installing, accessing, or using the Microsoft Kinect for Windows Software Development Kit (SDK), you agree to be bound by the terms of its license. Please read the Microsoft Kinect for Windows SDK license provided as part of this software.
See System Requirements.
- Make sure the Kinect sensor is not plugged into the USB port on the computer prior to running setup.
- If you are upgrading from v1.0: The SDK and runtime are upgrades over the v1.0 release. If you have the v1.0 SDK installed, the v1.5 SDK setup will uninstall it and replace it with the v1.5 SDK. The SDK, driver, and runtime are all fully compatible with v1.0. That means a Kinect for Windows application built using the Kinect for Windows SDK v1.0 should continue to run after installing the v1.5 SDK. If you encounter compatibility problems (other than the known issues listed below) please report them.
- If you are using a pre-release or Beta version of the Kinect for Windows SDK:
- You must uninstall any pre-release or Beta version of the Kinect for Windows SDK prior to running setup.
- If you have the Microsoft Server Speech Platform 10.2 components installed, you must uninstall them before installing this SDK. If you don’t uninstall them, setup will display an error. Note: This component was installed by Beta 2.
To install the SDK and Toolkit
- Make sure the Kinect sensor is not plugged into the USB port on the computer.
- If you have the Kinect for Windows v1.0 SDK installed, close any open samples, the Sample Browser, etc. You do not need to uninstall the v1.0 SDK. Skip to step 5.
- Remove any other drivers for the Kinect sensor.
- If you have Microsoft Server Speech Platform 10.2 installed, uninstall the Microsoft Server Speech Platform Runtime and SDK components including both the x86 and x64 bit versions, plus the Microsoft Server Speech Recognition Language – Kinect language pack.
- Close Visual Studio. You must close Visual Studio before installing the SDK and then restart it after installation to pick up the KINECTSDK10_DIR environment variables that the SDK requires.
- From the download location, double-click on KinectSDK-v1.5-Setup.exe. This single installer works for both 32-bit and 64-bit Windows.
- Once this SDK has completed installing successfully, ensure the Kinect sensor is plugged into an external power source then plug the Kinect into the PC’s USB port. The drivers will load automatically.
- The Kinect sensor should now be working correctly.
- To install the Toolkit, from the download location double-click on KinectDeveloperToolkit-v1.5.0-Setup.exe. This single installer works for both 32-bit and 64-bit Windows.
- To execute tools or samples, install samples, or explore other resources, start the Developer Toolkit Browser via the start menu: Kinect for Windows SDK 1.5 -> Developer Toolkit Browser v1.5.0 (Kinect for Windows).
If the driver is installed correctly:
- Look for the solid green LED on the Microsoft Kinect for Windows sensor.
- The Microsoft Kinect for Windows sensor will appear in Device Manager as the following nodes under “Microsoft Kinect”: Microsoft Kinect Audio Array Control, Microsoft Kinect Camera, Microsoft Kinect Security Control
- The Microsoft Kinect for Windows sensor's microphone array will appear under Sound, video and game controllers in Device Manager as: Microsoft Kinect USB Audio
If you have trouble getting the sensor or the SDK working, see Setting Up a Kinect Sensor.
The SDK Documentation is available online.
The Programming Guide section of the documentation has many important details for developers, including how to access the APIs from C++ and managed code projects, understanding the Natural User Interface APIs and Audio APIs, how to accomplish common tasks, etc. The documentation also contains API reference information. For additional help, see Additional Resources.
Online developer resources for Kinect for Windows include the following:
- Technical Resources on the Kinect for Windows Website
- The Kinect for Windows Human Interface Guidelines (HIG)
- The Kinect for Windows Blog
- Kinect for Windows SDK Forums
- Migrating from Beta 2
- Coding4Fun Kinect Showcase
For general Windows development questions, try the following:
- Microsoft Developer Network (http://msdn.microsoft.com): The latest development documentation to help you create applications for Windows.
- Microsoft TechNet (http://technet.microsoft.com): The latest administration documentation to help you administer the Windows operating system.
Changes in Developer Toolkit 1.5.1
Known issues for SDK 1.5/Developer Toolkit 1.5.1 (June 2012)
- We made the following performance and stability improvements to Kinect Studio 1.5.1:
- Reduced CPU usage overhead.
- Improved toolbar icons in Normal and High Contrast mode.
- Eliminated distortion and flickering that occurred in the Color Viewer display during playback using some video cards.
- Fixed hanging that occurred using AMD Radeon HD 5xxx series graphics cards.
- Fixed crashing that occurred during playback of .xed files that were recently saved to a network share.
- We made the following improvements to the Face Tracking feature:
- The Microsoft.Kinect.Toolkit.FaceTracking project now automatically copies the appropriate native libraries to the output directory in a post-build step.
- The FaceTracking3D-WPF sample now builds all configurations.
- We made the following fixes to the Samples:
- Audio Explorer – Fixed label captions that were clipped.
- Avateering – Reference to Microsoft.Kinect.dll is no longer version-specific.
- Kinect Explorer – Fixed crash that occurred in locales where “,” is used as the numeric separator.
- Polished the user interface in some samples.
- We made the following enhancements to the documentation
- SDK documentation is now available offline. For details see Kinect for Windows SDK Offline Docs.
- We tested SDK v1.5 and Toolkit v1.5.1 on top of Windows 8 Release Preview and found no new problems. Although that is not yet a supported production platform for Kinect for Windows, we continue to track well towards supporting it sometime after Windows 8 ships.
Changes in SDK 1.5/Developer Toolkit 1.5.0
Known issues for SDK 1.5/Developer Toolkit 1.5.0 (May 2012)
- As of this release, the SDK has been divided into a core SDK and a developer toolkit. These are two separate installs.
- All Samples have been moved into the toolkit
- We’ve continued significant sample investments in v1.5.0. There are many new samples in both C++ or C#. In addition, we’ve included a “Basics” series of samples with language coverage in C++, C# and Visual Basic. To explore the list of new and updated samples, please launch the Developer Toolkit Browser and explore.
- We’ve taken KinectSensorChooser, formerly part of the WpfViewers and split the logic and UI into two different classes: KinectSensorChooser and KinectSensorChooserUI in Microsoft.Kinect.Toolkit.dll.
- KinectSensorChooser could be used in non-WPF scenarios as it is logic only, no UI.
- KinectSensorChooserUI (used with a KinectSensorChooser instance) is a WPF experience that has undergone significant improvements in user experience. ShapeGame has been migrated to use it.
- Kinect Studio must be used in conjunction with a Kinect for Windows application. Start the application, then start Kinect Studio, and you can then playback and record clips. When you play a clip back, it will be fed into the application as if it was live Kinect data.
- Kinect Studio puts additional load on your machine and it may cause the frame rate to drop. Using a faster CPU and memory will improve performance.
- Your temporary file location (set via Tool/Options) should not be a network location.
- See known Issues section below
- It is now possible to receive skeletal tracking data when the Kinect camera is in the Near Range Setting.
- This setting is disabled by default to ensure backwards compatibility with 1.0 application
- Enable it through the SkeletonStream.EnableTrackingInNearRange property (managed code), or by including the NUI_SKELETON_TRACKING_FLAG_ENABLE_IN_NEAR_RANGE flag when calling NuiSkeletonTrackingEnable (native code).
- This works with both SkeletonTrackingMode set to Default or Seated.
- We suggest to use the Seated mode when using Near Range, since in most scenarios the player body will not be entirely visible. Ensure the torso and the head of the players are visible for locking.
- This mode tracks a 10-joint head/shoulders/arms skeleton, ignoring the leg and hip joints.
- It is not restricted to seated positions; it also tracks head/shoulders/arms when you are standing.
- Seated mode has been added to Skeletal Viewer (C++) and Kinect Explorer (C#). To try out the mode to understand its tracking ability, launch one of those applications and change the tracking setting from Default to Seated.
- For information on enabling Seated Mode, see the docs under Programming Guide -> Natural User Interface
- Seated mode skeletal tracking has higher performance requirements than Default mode, especially when tracking two players. You may notice reduced frame rate depending on the configuration of your system.
- KinectSensor.MapDepthFrameToColorFrame performance improvements: this the performance of this operation has been significantly improved with an average speed increase of 5x
- Depth and color frames will now be kept in sync with each other. The Kinect for Windows runtime will continuously monitor the depth and color streams and ensure that there is minimal drift between them.
- In managed code you will see that the frames returned from the KinectSensor.AllFramesReady event will have been captured at nearly the same time and will have timestamps that verify this.
- The RGB camera stream quality has been improved for the RGB 640x480 @30fps and YUV 640x480 @15fps video modes.
- The image quality is now sharper and more color-accurate in high and low lighting conditions.
- Kinect for Windows runtime provides Joint Orientation information for the skeletons tracked by the ST pipeline.
- The Joint Orientation is provided in two forms:
- A Hierarchical Rotation based on a bone relationship defined on the ST joint structure
- An Absolute Orientation in Kinect camera coordinates.
- This information can be used in Avatar animation scenarios as well as simple pose detection.
- Acoustic models have been created to allow speech recognition in several additional locales.
- These are runtime components that are packaged individually and are available here.
- Additional locales now supported:
- The Face Tracking component tracks face position, orientation and facial features in real time.
- A 3D mesh of the tracked face along with eye-brow position and mouth shape is animated in real time.
- Multiple faces can be tracked simultaneously.
- Face Tracking components can be used in native C++ code and a managed wrapper is provide for C# and VB projects.
- Bug fixes, stability, etc.
- Documentation improvements
- Improved documentation content
- The documentation is now online in MSDN Library. F1 help in Visual Studio will take you to API documentation online.
- The SDK Documentation CHM file is no longer distributed by setup. Online documentation URL is: here.
- Bug fixes, stability, etc.
Known Issues for SDK 1.5/Developer Toolkit 1.5.1
Installation of the DirectX SDK (June 2010) fails if installed after the Kinect for Windows SDK
There is a known issue with the Visual C++ 2010 Redistributable Package included in the DirectX SDK installer that causes this failure. You can find more information about this issue and the workaround at this location: Known Issue: DirectX SDK (June 2010) Setup and the S1023 error.
Kinect Studio 1.5.1
(These issues are issues that we plan to address in future updates to Kinect Studio):
In v1.5.0 and v1.5.1, Kinect Studio recordings (stored in .xed files) play properly only when the same Kinect sensor is being used by the application. You should keep a record of the precise sensor used to record each file so that you can play the file back with correct synchronization between color and depth cameras. With future updates, we plan to update the information that Kinect Studio stores in the .xed file so that playback is possible using any sensor.
If the Kinect-enabled application that you are recording turns skeletal tracking on or off while recording, recording stops.
If Kinect Studio ever crashes or hangs, the intact recording can be found in the temporary file directory specified under Tools > Options. (Change the file extension to .xed before playing it.)
SkeletonFrame.TrackingMode property is reserved for future use
- The TrackingMode property of the SkeletonFrame class always returns SkeletonTrackingMode.Default, even when the skeletal tracking (ST) engine is in Seated mode. To determine that ST is in Seated mode, applications can check the SkeletonStream.TrackingMode property or confirm that the TrackingStatus property of one of the lower 10 joints of a tracked skeleton, such as the HipCenter joint, returns NotTracked.
- The value returned by the SkeletonFrame.TrackingMode property may change in future releases. Applications should not use this property.
Do Not Use INuiSensor::NuiUniqueID or KinectSensor.UniqueKinectId
- Do not use the KinectSensor.UniqueKinectId property (or the corresponding INuiSensor.NuiUniqueId method) because of inconsistent behavior between Xbox and Kinect for Windows sensors. Instead, use the KinectSensor.DeviceConnectionId property to identify the sensor based on the USB port it is plugged into.
From the Kinect for Windows SDK v1.0 Release
- When StatusChanged events occur, ensure that you call kinectSensor.Start() on the transition to KinectStatus.Connected. Call kinectSensor.Stop() on all transitions away from KinectStatus.Connected. (Test with unplugging, unpowering, and machine sleep/hibernate.)
- Calls to KinectSensor.Map* and DepthImageFrame.Map* after you have called kinectSensor.Stop() (or a sensor has been removed while the application is running) may fail. Ensure that you protect against that case - use a try/catch around all calls to those Map* APIs.
- If C++ code is executing NuiInitialize/NuiShutdown multiple times through the application’s lifetime, SetDeviceStatusCallback should be called once, before invoking those calls.
USB host controller compatibility
The Kinect for Windows drivers have encountered compatibility problems with certain USB host controllers when tested in specific computer configurations. Some examples of these controllers are the AMD Fusion A75 USB2.0 controller, certain NEC USB 2.0 controllers, the Etron USB 3.0 Extensible Host Controller, or the Renesas Electronics USB 3.0 Controller. On those controllers, you may encounter a problem, such as a low RGB frame rate, or a Not Ready status when plugging/unplugging a Kinect. If this happens to you, you may have to restart your app or reboot your computer. You can also try updating your USB controller drivers, or plugging your device into another USB 2.0 controller.
Number and date normalization in Kinect for Windows Speech Grammars
- Kinect for Windows is now integrated with Microsoft Speech Platform Version 11, an upgrade from the Version 10.2 included in previous Beta versions. This version, in conjunction with the Kinect for Windows Speech Language Pack, provides superior speech recognition capabilities to Kinect for Windows.
- Note that this version comes without the support for standard TN/ITN (numbers and dates normalization), requiring some changes to grammars built on top of previous Beta versions. Now you need to expand numbers and dates on your grammars. For example you need to use “Play song number five” in you grammars, instead of “Play song number 5”.
Microphone Array default gain setting is sub-optimal
- The default gain applied to the sound recorded by the Kinect sensor’s Microphone array is higher than optimal and can affect speech recognition quality when users are speaking close to the Kinect sensor.
- If you notice the Kinect audio signal clipping in your application, or experience the Speech Platform SDK reporting AudioSignalProblemOccurred events use TooLoud audio input, set the microphone gain level to 3 on a scale of 100.
- To set the correct gain value:
- Open the Windows Control Panel, select the Sound icon, then the Recording tab.
- Select the Kinect Microphone Array and press Properties, select the Levels tab.
- Set the Microphone Array gain level to 3 (three), changing it from the default of 100. Press Ok and then Ok again.
Audio is not processed if skeleton stream is enabled after starting audio capture
- Due to a bug, enabling or disabling the SkeletonStream will stop the AudioSource Stream returned by the Kinect sensor.
- The following sequence of instructions will stop the audio stream:
kinectSensor.Start(); kinectSensor.AudioSource.Start(); // --> this will create an audio stream kinectSensor.SkeletonStream.Enable(); // --> this will stop the audio stream as an undesired side effect
The workaround is to invert the order of the calls or to re-start the AudioSource after changing SkeletonStream status.
Workaround #1 (start audio after skeleton): kinectSensor.Start(); kinectSensor.SkeletonStream.Enable(); kinectSensor.AudioSource.Start(); Workaround #2 (re-start audio after skeleton): kinectSensor.Start(); kinectSensor.AudioSource.Start(); // --> this will create an audio stream kinectSensor.SkeletonStream.Enable(); // --> this will stop the audio stream as an undesired side effect kinectSensor.AudioSource.Start(); // --> this will create another audio stream