instantreality 1.0

We have further improved picking to cope with several problems that came along with the original approach (which was mentioned in a very old post). Therefore, picking now supports 64k different objects, a higher precision pick position and the normal at the picked position (both in world space) for all mouse events.

We still use a single-pass render-buffer-based approach, but instead of rendering the normalized world position into an FBO’s 8-bit RGB channel and the (internal) Shape ID into the (also 8-bit) alpha channel, we now render just the distance of the picked object position to the camera position into the RG channel (encoded as 16-bit value in the shader) and the Shape ID into the texture’s BA channel (also encoded as 16-bit).

Having the distance d between both positions provides enough information to calculate the full 3D position, since the x and y components (along with z later on) can be obtained by computing the view ray through the the picked pixel position (x,y).

And instead of just reading back a single (8-bit) RGBA value at the picked pixel position (x,y), we now read back a small 2×2 window, so that we can also directly compute the object’s normal by taking the cross product of the (decoded) world space position above (x,y-1) and to the right (x+1,y).

This way, the corresponding UI Event object now not only provides the picked world position (worldX, worldY, and worldZ), but also normalX, normalY, and normalZ.