Converting video into geometric abstractions

So I am trying to come up with an effect that will turn a video into geometry.
I basically want to be able to track an object and transform that object
into geometry and composite it back into the video.

So if it was Brad Pitt on screen, you would see a crude geometric Bizaro Pitt.
It would be nice to be able to control the vertex count so I could make it
more detailed. Vertex depth/height could be controlled by pixel color and luminosity.

Though the principal is easy:
– run your video matrix through a luminosity converter (don’t remember the object name right now)
– run the lumo map into a jit.expr that takes the x and y coordinates of the input cells as output x and y, and takes the lumo (color) value for z
– run that into jit.gl.mesh position inlet and send the original video matrix into the texture inlet

you can also skip the first step and do the lumo calculation from the RGB values in jit.expr.

this reasoning might be a bit buggy but it’s the general direction you should be heading for

here’s a simple patch to get you started.
it uses jit.gen and jit.gl.pix to handle some of the minor things you need to do, like remap the xyz values, multiply the Z by luminosity, and rotate the texture properly.

let me know if you have any questions.

-- Pasted Max Patch, click to expand. --

Copy all of the following text. Then, in Max, select New From Clipboard.