Tag: Kinect V2

Another day closer to my international move and another weird task done: today’s one is get approved to get an international driving license. This is why today’s post is also a short one (that’s a shame, this topic is really cool!).

Our MVP friend Vangos Pterneas shared some time ago, a very powerful library which allows us to leverage the capabilities of Kinect V2 to perform hand ans finger detection, as always all the code on GitHub.

Vango’s post explains in detail how the detection works. I’m not very smart, but I think I understand the basis: Kinect will detect the the joint of the hand, and with this center point we’ll work in a radius of + 7 – 15 cms, and that’s the hand. From there, the library allows us to work with the hand context and also each finger separately.

The source code implementation to be used is also very cool. Once a frame is processed, we can ask the detection engine to find hands and once something is detected, we’ll have information about the contour and the collection of fingers to work with them.

While I hope that fall into my hands a Surface Book or a Surface Pro 4, one of the new features that wanted to try was Windows Hello. Windows Hello, is part of a new set of libraries on Windows 10 [Windows.Devices.Perception]

Within this namespace we can find a series of UWP APIs that allow us to access information that we already know developing for Kinect as depth, infrared, color cameras, and metadata. With this information, we have a solid base to perform actions of Computer Vision and enable the identification process using faces, that is Windows Hello.

The new Microsoft Phone, 4 Surface and Surface Book, already includes the necessary hardware to work these features cone. There are also cameras certified Intel Real Sense F200, to access Windows Hello.

Kinect V2 is not a certificated device (yet), but there are some tricks that we can use it as part of Windows Hello. For this we have to enable the distribution of test drivers for Windows 10. This is accomplished with an entry in the Windows registry:

One of the big differences between Kinect V1 and V2 is that in version 2 we can use XBox One Kinect with our computer without any problems. If we go back a little, with the Kinect V1 we used to had 2 versions

Kinect for Windows

Kinect for XBox

Officially, you could use the Kinect for XBox to develop apps, but then the production stage you should use the version for Windows.

With version 2, this has changed, and if you have a XBox One with Kinect, you can use it with Windows. And of course, you need this adapter: Adapter Kinect for Windows.

When I get the box I got scared a bit, since it is quite big

Then quickly startle is passed, there are lots of air inside the box and 4 wires of rigor for the Kinect

When you work with apps on Kinect that require a special code for each body recognized by the sensor, the entry point tends to be the collection of bodies returning to us in the line 15 of the first block in the following code example

The problem with these lines is the collection of bodies it is usually complete by 6 elements, however there are no 6 people in front of the Kinect. The following image shows the collection and at the same, the 6th element is a correct body, but with the property IsTracked = False.

The solution is quite simple, a small array preprocessing, filtering by the bodies that are correctly identified. In line 16 of the second block of code we obtain the total number of bodies where IsTracked == True and from there to be processed.

A long time ago, a very interesting tool was added in KinectSdk version 1.5: “Kinect Studio” (I wrote a review 2 years ago here). This tool allows us to record Kinect sessions including all sensor captured information, like camera feed or infrared feed. If you develop Kinect apps wich require very repetitive testing, this type of recording is very good, since with them we avoid us having to put us in front of the sensor and do again and again the same gestures.

The problem in the Kinect V1 SDK is that you had to have a Kinect V1 SENSOR connected to the computer where you record and play KINECT STUDIO. This force to have a Kinect for each developer. That’s why because Kinect Studio was a sort of hack and “strained” the feed from the sensor usb where it was connected the Kinect to send information.

The new Kinect V2 SDK has changed that in a great way. Now is not the USB which sends information of the Kinect, instead there is a Windows service which manages the information registered from the sensor and the service publish this information. So, taking advantage of this, the new Kinect Studio allows us to be able to use Kinect Studio without a Kinect V2 sensor.

The following animation shows app example “Body Basis” included with the SDK on a computer without a connected V2 Kinect sensor, and how to open a recording Studio Kinect, connect it, the app starts to display information as if the sensor was connected.

In cases where to touch you to work with colleagues in remote and complicated ‘share’ the sensor, this is an excellent choice 😀