Hierarchy, gluProject and collision detection

I'm starting a new job in January, and I am working on a project to pass the time. I've been teaching myself how to use OpenGL with some success, but I've come against a wall.

I am trying to simulate a particle-based deformable object. The deformation itself is working as expected, but I am having some trouble with collision detection of the deformed object against solid objects (a cube for now). I know that the source of the problem lies in the fact that the objects that are colliding are hierarchically modeled, so they use different transformation matrices. I tried to get around that by calling gluProject against a particle that I know is colliding with the solid cube, but gluProject gave me nonsensical window coordinates.

Here is the code that I use to call gluProject. coords is a 2D array containing 3D coordinates for each particle.

I guess I'm not sure what you are trying to accomplish. If you have one 3D object(the cube) and another 3D object (the deformable mesh), and you want to find out where they collide, why do you need window coordinates?

Sorry, I skipped a step in my explanation. The idea is to project the particle to the window, then the window to the cube. I don't know the proper method for converting the coordinates of one object to the coordinate system of the other when both are in arbitrary positions on a hierarchical model.

After looking at this for a while, I think I have found a better solution to my original problem. I think I should be able to use the projection matrix to convert between two projections. (I probably should have thought of that first, but I'm still acquainting myself with OpenGL's matrices.)

I am still curious about the reason for gluProject's inaccuracy. gluUnproject works fine for me, so I can't imagine what I could have done wrong for such a similar function.