Depth and Color image do not match exactly

Depth and Color image do not match exactly

I'm trying to match depth image onto the color image using PXCImage::ImageData.planes[2] component of the depth data that contains coordinates of a depth point in a color image. While it seems to do the job, I notice a significant misalignment. I attached a rather crude picture where I blacked out all pixels which have a depth bigger than a threshold.

The depth map does not exactly match the color image, actually the shift to the same side where the depth sensor is located on the camera with relation to the color sensor. Is this a deficiency of SDK?

I tried median filter, but it didn't really help, because the errors were not in the depth map, but actually in the mapping between depthmap coordinates and image coordinates. I tried plain and simple x'=x*2, y'=y*2, where (x,y) and (x',y') are coordinates in the depth map and the image respectively because the image was of the size 640x480, I've got no more false positives, but of course a huge misalignment, like in the previous example. Is there any way to properly map between depth and color images?

Allegati:

So I have started to work on an geometrical solution to the problem of aligning depth and colour images exactly, but just to make a note - does it really mean that currently there is no calibration of two sensors that will provide exact mapping in the SDK? Or is it just a temporary problem in the Beta version? Thanks.

So I have started to work on an geometrical solution to the problem of aligning depth and colour images exactly, but just to make a note - does it really mean that currently there is no calibration of two sensors that will provide exact mapping in the SDK? Or is it just a temporary problem in the Beta version? Thanks.

Hi Xintian, thanks for pointing out, I've chosen the wrong method to demonstrate the issue. And actually half of the problem doesn't exist, the mapping is indeed pretty accurate except the fact that there is some sort of mirroring going on to the right side of the face.