Milkymist is Digital Visual Synthesizer and Processor, Built as Sophisticated Open Source Hardware

Even before we get to the “open source hardware” angle, it’s special just finding something that can function as a self-contained, digital visual instrument. And Milkymist more than qualifies, as a video synth that can produce seemingly-endless generative outputs from raw image and live video inputs.

But it is significant that this is open source hardware. Milkymist in its present form isn’t for everyone, but it’s evolving fast – and it’s a visual coder’s and hacker’s dream. Entirely custom hardware runs elegant scripts for effects like video feedback with razor-thin low latency and blazing framerates and performance. Add the open source spirit to that, and apart from the entire design and software being open to your inspection and modification, you get a box that wants you to add inputs, unusual controllers and sensors, and generally muck about with what you can do.

I’ve been watching this project for some time, but had to fully wrap my head around it. One of the developers, Sébastien Bourdeauducq, had been nagging me for some time. “Yes, yes, it’s open source, it does … things,” I responded. “Great.” But to tell the truth, I hadn’t fully grasped what would make you go down this path rather than a computer. Finally, I gathered Sébastien and some friends in a flat in Berlin-Friedrichshain to have a look.

It’s when you see the speed and low-level hackability of this thing that you get it. Jack in a keyboard, and just a couple of lines of code in the Milkymist’s scripting language, and you can suddenly produce an eye-popping feedback effect or layer. Being this close to the hardware is just terrific: you switch on the device, write very lean scripts, and it just works. And while you could do the same with a computer, there’s no mucking about with drivers and trying to contend with high input-to-output video latencies. Nor do you have to set up a whole environment before you begin scripting – it’s there right when you turn on the device (see screenshots of the interface.) There’s even a camera included in the box so you can get to work right away.

Granted, this kind of number-crunching with pixels may not appeal to you non-coders. So the main test of Milkymist in 2012, I think, is whether a combination of flexible presets, controller mappings, and documentation can turn this into something other people can just pick up and play. I was impressed enough that I’ll be loaning a unit to try my hand at that, alongside more conventional computer-based projects.

And yes, of course, a computer GPU has greater capabilities. But at this price, and with this integration, I think it’s best to think of Milkymist as a hacker-friendly, high-performance, standalone instrument. Hopefully soon we’ll also see mixer that make it easier to integrate a computer and something like the Milkymist. (There are options for doing that now, but not the digital mixer of which we’re all dreaming.)

Inside the Hardware

See the unopening and hardware tour above. Just a few of the specs that make this worth a look:

This is all built on an FPGA, or “Field-programmable gate array.” For those of you are aren’t integrated circuit geeks, what that means is that you have a hardware platform where you can actually customize the function of the hardware after you’ve got it, not simply by changing firmware on a chip, but actually modifying the logic of the circuitry. (The Borg would have loved this.) It’s some hardcore stuff, but it’s about time a creative open source hardware project took advantage of this kind of platform, as it seems uniquely suited to the open source hardware concept. And what this means for you as an end user is that the Milkymist’s capabilities – thanks to all this custom hardware and software – are constantly improving. It’s hardware that gets better rather than worse; it’s the opposite of planned obsolescence.

Deep configurability – getting right at pixels and frame buffers with scripting, from the moment you turn it on, and being able to re-configure the hardware via the FPGA – is really the soul of the tool. You can re-program the very way in which signal is processed inside the box. And, in this case, the primary advantage of the FPGA is democratization of custom hardware, enabling a small band of coders to brew their own device, as with the Milkymist.

In Action

Milkymist hasn’t been seen out in the wild much, though hopefully that will change shortly. Here’s an example with our friend, the tallented visualist NO CARRIER, playing in Philadelphia at 8static in November, with Cheap Dinosaurs. (Skip ahead in the second vid – it gets good – or check out the picture gallery on Flickr.)

And lastly, if you’ve got some time, here’s the complete video of an extended presentation at Berlin’s SCOPE Sessions, which is becoming a valuable hub for the visual scene:

We’ll be following this story more, so if you’ve got more questions (including “wait, really, what is this for anyway?”), we’ll entertain them. Stay tuned for some hands-on details shortly.

this. is. super. cool.
it seems to be custom made for exactly what i need to do! i suppose there are other software possibilities out there, but it is nice to have something that is completely dedicated to what i want to do visually right away, and with the ability to just get in there quickly at a low level….

Lau

looks super cool.
wonder if the functions of the fairlight CVI could be copied/emulated on this device. Had the real thing for some months some years back. Loved the thing.