Movement on the surface of a pool

Hi everyone, I’m chewing over a potential project for the summer that would involve extracting information from a swimming pool. The idea is that participants create music and localized visual projects by splashing. I’m thinking that performing motion tracking on the waters surface from above might be one way to achieve this but I’m unsure how effective it would be. The project is aimed at young physically disabled participants and the idea is that their x-y position in the pool is observed from above and that when they create ripples or splashes that this will create audio and localized visual projections that vary depending on their position in the pool. At the moment I just looking for ideas and objects that might be worthy of exploration. Having underwater microphones is also a possibility. Many thanks in advance!

Yeah I was thinking frame to frame delta, I’ve used that before and the jitter cookbooks offer pretty sturdy patches. I worry that the visual projections on the surface of the water will create a feedback loop. I’m guessing that infrared lighting and cameras with the infrared filters extracted might offer a workaround but I have not got a huge amount of experience in that regard.

Thats certainly a possibility. In my mind, seeing it from the POV of the participant, you would get colour playing off the surface of the water around you, not distinct images – just streaks of abstract colour localized around you, thats kind of the effect I’m thinking of. This is all very much up in the air at the moment, I haven’t even seen the pool yet! ;)

Maybe once you have some color info from the pool, you can specifically eliminate that color from what’s coming out of the projector (it sees and tracks blue to bluish-white but only spits out purple, red, green etc.) This should eliminate most feedback problems. With some timed fades of your camera input/colors you should be able to jockey the input/output. And I think that even though a lot of the projector light energy would go to the bottom of the pool, it would still look good, and any local splashing or disturbance would make it look really dynamic and cool.

Maybe if the participants also had attachments to their arms (like styrofoam "fins" or something) that would give the camera something else to pick up and the projector something else to project upon.

Really interested in what you come up with, and whether it works well without infrared cameras. Not that they’re bad (in many cases absolutely necessary) but if you can use more standard stuff for something, that’s always good…

the first thing what comes to mind is that there are a shitload of different
numbers which you can generate by measuring the _distance between
the persons.
3 values from 3 persons, 6 from 4 persons and so on.

the speed and direction of their movement is also relatively
easy to capture and to transform into useful data for audio and video.

In terms of tracking the participants in the water, I need this system to be as rugged as possible. Predictability is a big thing when working alongside some participants with special needs and want to minimise random actuations of sound and video material. If we are filming from above, in infrared, projecting images onto the pool that are localised to participants, what would be the best ways to extract useful location and movement information? As mentioned earlier frame to frame delta is a possibility, any other objects or techniques that anyone can think of? Ultimately it would be nice to get good sturdy x/y coordinates from each participant and data regarding the level of movement expressed by each participants i.e, you splash a lot, you get a lot of localised video and sound, you don’t splash a lot, you don’t get much of either. Could cyclops offer good results for tracking? I downloaded the trial several months ago but foolishly didn’t have time to use it that much and consequently I’m not that familiar with all of its capabilities. cv.jit? Is frame to frame delta the best way of seeing and evaluating levels of movement? Many thanks for you kind support and brilliant ideas!