Google, Danger Mouse Team for Steerable 3D Video-in-a-Browser

Tablets and smartphones may be all the rage these days, but here’s something computers can do that they can’t, for now anyway: display an interactive three-dimensional environment like the one in “3 Dreams of Black,” a browser experience from Google and musician/producer Danger Mouse and directed by Chris Milk.

The interactive film brings together much of the team behind the incredible Google/Arcade Fire HTML5 project of last August to showcase what can be done with WebGL, a new technology for the browser that lets websites use your computers graphics card — and judging from an early demonstration on Wednesday, the project has spawned an entirely new way to experience music in a browser.

This “interactive film” takes place in a three-dimensional environment (similar to the ones you see in modern videogames, as opposed to 3D in the sense of televisions that require glasses) using WebGL, a new web standard embraced by Chrome and FireFox a couple of months ago, which lets web developers harness a computer’s graphics card to display rich environments.

Your PlayStation doesn’t need to be worried — yet — and your tablet can only dream of doing this, for now, but the technique allows a computer browser to display complex graphics that react to mouse movements, allowing a user to move or look around in three dimensions the way one can in some videogames.

The project is not yet online at ro.me (update: it is now), as the team is still putting finishing touches on a few details. Google Creative Lab technology lead Aaron Koblin demonstrated parts of “3 Dreams of Black,” which accompanies the Danger Mouse and Danielle Lupi song “Black” from their spaghetti-western-inspired Rome record, at the Google I/O conference in San Francisco on Wednesday.

His demonstration (screenshots throughout) included a browser-based character modeling tool with which amateur computer artists can create animals and other elements for inclusion in the game. People will vote on these items, and the most popular ones will be incorporated into everyone else’s experience of the film. And all of these tools are open-source, the code freely available for other developers’ experimentation.

As for the storyline, “3 Dreams of Black” (produced by @radicalmedia with animation by Mirada) is told from the point of view of a little girl who goes to sleep and has three dreams, each with its own interactive “rabbit hole,” explained director Chris Milk.

Regular users can create new elements for inclusion in other people's experience of the film using this Google-created, open-source, browser-based tool.

“It was two different things coming together,” Milk told Evolver.fm via phone on the eve of the announcement, explaining how the project came about. “I did the Arcade Fire’s piece, ‘ The Wilderness Downtown,’ with Google, and that was what we call a Chrome Experiment… Separate to that project and that relationship, I was developing another narrative project with Danger Mouse (a.k.a. Brian Burton) and he’s a producer behind Gnarls Barkley, Broken Bells, and a zillion other bands.”

Danger Mouse and collaborator Danielle Luppi had originally planned to include still photographs from the imaginary movie to which Rome would be the putative soundtrack. Milk convinced Danger Mouse/Burton that it should be possible — not to mention worthwhile — to make something with moving images to accompany the cinematic music of Rome, and thus, this concept of telling a longer “transmedia” narrative was born.

“I have a philosophy about telling stories through multiple channels, and how we as humans are far-more-sophisticated media-consuming beings than we were ten years ago,” said Milk. “We can actually track these stories on multiple platforms for longer periods of time.”

“3 Dreams of Black” uses WebGL to let us do exactly that, weaving 2D photos, 3D environments, and user control so that the experience never quite unfolds the same way twice. Not only can you look around, but at certain points, your actions determine where you go next, and the whole thing is riddled with surprises.

“There are a lot of ‘easter eggs’ and hidden things, and at the end of it… there’s a lot to explore,” said Milk. “It may seem at first like it’s just one three-and-a-half-minute-long piece, but if you keep looking and going further, there’s no shortage of rabbit holes to go down.”

“It is very much like a lucid dream,” explained Koblin to Evolver.fm, also via phone. “It starts out with her waking up and going through this journey, passing through a city scene, going on a train ride, and into some prairie-scapes — just a very lucid experience.

“It’s being guided by the experience itself, and all the visuals are being presented,” he added, “but you have a sense of how it plays out controlling this ‘soup’ that’s flying throughout 3D landscapes and colliding with buildings and atmosphere, like sand dunes.”

“[The 'soup' surrounding the character] is interacting with the elements that are there, and also leaving traces, and there’s crazy wildlife that’s springing forth from the soup. There’s a light soup with birds and greenery and a dark soup with bears, wolves, spiders and things like that.”

Milk said this impressive-sounding next-generation music video for the browser, set to be released tomorrow or shortly thereafter, is the first installment of a longer narrative along the lines of Pink Floyd’s The Wall in scope and ambition — “at least a music video and live tour,” and possibly “a long-form feature film” — so stay tuned, and not only for segues to this project. Other bands will likely utilize WebGL to create their own experiences, too.