Video: Robot Turns Your Dreams Into Art

Ever wondered what your nightly tossing, turning, and snoring would look like if turned into art by a robot? Now you can find out, by winning the chance to stay at a hotel belonging to European chain Ibis, where an ABB robot paints your sleep patterns.

As part of the hotel's "Sleep Art" advertising campaign, ABB's smallest robot, the IRB 120, will paint patterns inspired by the dreams of 25 people. By sleeping on a specially designed mattress in an Ibis-owned Paris, Berlin, or London hotel, participants' dreams will be conveyed to the robot via 80 motion, sound, and temperature sensors in the mattress' topper.

Winners of a chance to stay at a European hotel chain will leave the next morning with a brand new piece of art, based on their sleep pattern data, which will be captured by sensors and painted by this ABB robot. (Source: ABB)

"The topper is built with a grid of force sensing resistors (FSRs), which detect exactly where and how the person is lying," ABB France spokesman Guillaume Pradels wrote in an email to Design News. "Over time, this obviously translates to motion as well."

The sensor grid is connected to the "Dreambox," which collects detailed information on each individual FSR. Along with the room's temperature and sound levels, also captured by the Dreambox, this data is then streamed by a WiFi link to a computer that controls the art production.

The software running on the computer, and the Dreambox hardware, was made by ACNE Production, the Swedish production company overseeing the project. After receiving the sensor data, the computer parses it into an algorithm and sends the sleeper's positions to the IRB 120.

ABB wrote software that collects the data from the Dreambox and allows the robot to capture sleep patterns, process the data, and turn it into a personalized visualization. ABB also wrote software that talks to the IRB 120's controller interface using a TCP socket via the IP protocol over Ethernet. After the IRB 120 gets the data from the computer, ABB's robot path algorithm calculates the correct path by applying the relevant orientation on the brush, avoiding collision, and other out-of-reach or singularity problems.

Proprietary smart path computation assures the robot's delicate and smooth movements. So does the fact that the robot path is speed-independent, due to ABB's proprietary QuickMove and TrueMove technology. The result is a series of multi-colored brushstrokes applied to a canvas in a rendering of the data that becomes a unique acrylic painting.

In its normal everyday life, the IRB 120 is used mostly in the 3C (computing, communication, and consumer) industry for handling parts, dispensing, and machine tending. It was chosen for the Sleep Art project for several reasons. Both the manipulator and the controller, which each weigh 25 kg (55.1 lb), are small in size and weight, so it's relatively easy to handle. The robot also has 0.01mm top-of-class repeatability, as well as top-of-class accuracy.

Although digitally streaming computed data to an industrial robot is not new, the robot path algorithm developed for this application is unique, said Pradels. ABB hasn't decided yet what other applications the algorithm might be used for. One possibility might be digital fabrication in construction, which has already been explored by architects Gramazio & Kohler using ABB's IRB 4600 robot.

It is an interpretation of the data, so how was the software designed? It would be interesting to see how the data is handled. I will admit, I'd like my dreams to be interpreted. I would like to see a nightmare vs. a typical one.

Robert, I agree. Ann this is a nice article.The idea of using FSR data to control a robot is pretty cool. I discussed FSR's and motion control in my book, Learn Electronics with Arduino and can see the technique in operating a robot being implemented in this application. I'm wondering what's the room rate for this techno-art experience?

Cabe, yes it would be interesting to see how the sensor data is converter to robot movement. Since the data is coming from body movement, it may not distinguish between pleasant dreams and unpleasant dreams unless the dream affected body movement.

I know when my wife is having a bad dream, she tends to toss around and mumble. If the data is body sensors and audible, could the software interpret 'erratic' behavior? Or how about erotic behavior? Or most nights, you remember nothing?

Either way, this is cool. Wake up in the morning and see what surprise painting is waiting for you!

I agree about the interpretation of the data: in fact, that was my first (and second and third...) question to ABB: what were the assumptions in the software design about how motion, temperature and sound sensor data would be interpreted visually? Although I didn't get an answer, it's obvious that you can design it any way you want (more or less). So the applications could be pretty broad.

Focus on Fundamentals consists of 45-minute on-line classes that cover a host of technologies. You learn without leaving the comfort of your desk. All classes are taught by subject-matter experts and all are archived. So if you can't attend live, attend at your convenience.