Overview

As previously mentioned, the OpenNI API is written in C++ but once you follow the installation procedures covered, here, you will have some pre-compiled wrappers that will give you access to use OpenNI in a few other languages if you need.

Since there is a plethora of existing tutorials regarding setting up various development environments in C++ and corresponding example projects, this article will show you how to setup the 32-bit OpenNI JAR (OpenNI Java wrapper) in Eclipse IDE. We will then initialize an OpenNI production node to begin accessing Kinect data and to get the RGB stream into OpenCv, which is a popular computer vision library.

Before going on to the following project, make sure that you have all of the dependent libraries installed on your machine. For the instructions on getting the 3rd party libraries and for setting up the development environment check out this post.

Also, I want to clarify that this code is merely one solution that I managed to successfully execute. This said, it may have bugs and/or mayb be done more successfully or more easily using a different solution. If you have any suggestions or find errors, please don’t hesitate to contact us and I will change the post immediately. These posts follow and continue to follow exploration and collaboration.

Using the Kinect’s RGB feed

In this project we will:

Make a simple program to capture the RGB feed from the Kinect in Java

Get the data into an OpenCV image data structure

Display the data on the screen

A high-level overview of the steps we need to take are as follows:

Create a new ‘context’ for the Kinect to be started

Create and start a ‘generator’ which acts as the mechanism for delivering both data and metadata about its corresponding feed

Translate the raw Kinect data into a Java data structure to use in native Java libraries

Eclipse will also fill out our “main” statement for use by checking the box on the project set up. One addition we will need to make is to surround the following code block with a statement for any exceptions that may be thrown when starting the data feed from the Kinect. Here we are starting a new “context” for the Kinect

We also need to pick the pixel format to display from the Image Generator. We will use the Red-Green-Blue 8-bit 3 channel or “RGB24″

imageGen.setPixelFormat(PixelFormat.RGB24);

Sample RGB Project – Part 3

OpenNI also allows us to easily mirror the image so movement in 3d space is reflected in the image plane

context.setGlobalMirror(true);

Create an Iplimage(opencv image) with the same size and format as the feed from the kinect.

IplImage rgbImage = IplImage.create(imWidth,imHeight, 8, 3);

Next we will use the easy route and utilize JFrame/Javacv optimized canvas to show the image

CanvasFrame canvas = new CanvasFrame("RGB Demo");

Now we will create a never-ending loop to update the data and frames being displayed on the screen. Going line by line we will, update the context every time the image generator gets new data. Set the opencv image data to the byte buffer created from the imageGen.

NOTE: For some reason the channels coming from the Kinect to the opencv image are ordered differently so we will simply use the opencv convert color to set the “BGR” to “RGB”. We tell the canvas frame that we created to show image.

Sample RGB Project – Part 4

Finally, we need to also release the Kinect context or we will get an error the next time we try to start a node because the needed files will be locked

Primary Widget Area

This theme has been designed to be used with sidebars. This message will no
longer be displayed after you add at least one widget to the Primary Widget Area
using the Appearance->Widgets control panel.