Photoshop: Smart obj stack mode blend to video

I'd like to be able to apply a smart object stack mode directly to a video. It would use the videos frames as layers in the smart object. This can only be done by converting a video to frames and then converting it and applying a stack mode. It's far from automatic and the video import is limited to 200 frames/layers. My last smart object used 2500 layers from a video. The results were fantastic but could be improved in both the result and the efficiency. I'm sure there are many uses for this, I used it to show the movement of people through a public space for architectural work. Thanks

Please do, I'd love to see an advancement to this sort of image processing. Heres an example I have worked with, 2500 layers from a 30 minute hd video, would prefer more layers but that will do for now as the results wouldn't have made that much difference for my end result. Not a brilliant result do to lots of factors such as the time of day it was shot which wasn't good for the sunlight etc...

I later when on to skew the image to imitate a plan view and went on to work in 3DS Max applying it as a displacement map to a simple plane. This allowed me to generate 3D forms directly from the video of people walking. i.e. made the areas which people did walk lower and the areas where people didn't walk higher in my case.