YouTuber KOALAB looks like a mad scientist in his latest video experimenting with Mixed Reality MIDI controllers. Here he combines several different technologies across macOS and iOS. The hardware is also a mix of off-the-shelf gear like Leap Motion, with off-the-fucking-wall gear like the $400 Bridge helmet for iPhone, and even some custom hardware for monitoring hand rotation. All tolled this experiment probably cost a small fortune. So it is unfortunate that the setup suffers from noticeable latency; while flat-out ignoring some of his commands!
Still this gets the discchord Stupidtacular seal of approval for sheer inventiveness.

Video Description:

It is a virtual MIDI console that can be used with the same sense as a hard MIDI controller.I applied for GUGEN 2017.

Reader Comments 1

The concept is pretty neat. It does sound like one key is to map out controllers to specific gestures. Reminds me a lot of Leap Motion, which has been making a pretty big push for AR/VR.But, as you say, this demo suffers from the “close but not quite” problem. Can be inspiring, but the current implementation doesn’t sound that useful.Those types of project keep cropping up, so it must be “in the air”, what with the VR hype and people’s hopes shifting to AR. Does sound, yet again, that Augmented Reality will have a broader impact that Virtual Reality.

Wonder how many people are experimenting with similar things. University departments probably have too many of these projects to count.