Google’s DeepMind AI gives robots the ability to dream

Following in the wake of recent neuroscientific discoveries revealing the importance of dreams for memory consolidation, Google’s AI company DeepMind
is pioneering a new technology which allows robots to dream in order to
improve their rate of learning. Not surprisingly given the company
behind the project, the substance of these AI dreams consists primarily
of scenes from Atari Video games. DeepMind’s earliest success involved
teaching AI to play ancient videos games like Breakout and Asteroids.
But the end game here is for robots to dream about much the same things
humans do – challenging real world situations that play important roles
in learning and memory formation.

To understand the importance of dreaming for
robots, it’s useful to understand how dreams function in mammalian minds
such as our own (assuming the ET readership doesn’t include any aliens
eavesdropping on the tech journalism scene). One of the primary
discoveries scientists made when seeking to understand the role of
dreams from a neuroscientific perspective was that the content of dreams
is primarily negative or threatening. Try keeping a dream journal for
a month and you will likely find your dreams consist inordinately of
threatening or awkward situations. It turns out the age old nightmare of
turning up to school naked is the rule rather than the exception when
it comes to dreams. Such inordinate negative content makes little sense
until viewed through the lens of neuroscience. One of the leading
theories from this fields posits that dreams strengthen the neuronal
traces of recent events. It could be that negative or threatening
feelings encountered in the dream help to lodge memories deeper into the
brain, thereby enhancing memory formation. DeepMind is using dreams in
a parallel fashion, accelerating the rate at which an AI learns by
focusing on the negative or challenging content of a situation within a
game.

So what might a challenging situation look
like for a robot? At the moment the world’s most sophisticated AI’s are
just cutting their teeth on more sophisticated video games like Starcraft II
and Labyrinth, so a threatening situation might consist of a
particularly challenging Boss opponent, or a tricky section of a maze.
Rather than pointlessly rehearsing entire sections of the game that have
little bearing on the player’s overall score, “dreams” allow the AI to
highlight certain sections of the game that are especially challenging
and repeat them ad nauseam until expertise is achieved. Using this technique, the researchers at DeepMind were able to achieve an impressive 10x speed increase in the rate of learning.

A snapshot of the method published by the DeepMind researchers to enable AI “dreams”. Image courtesy of Deepmind.

You might ask why AI “dreams” are necessary
given that robots can already dominate humans in most games such as
Chess and Go. To grasp this, it is necessary to differentiate between
AIs that use supervised vs. unsupervised learning. Most of the
impressive feats so far attained by AI have been made using supervised
learning, in which organized “training data” is supplied by the
programmers and the AI learns to detect patterns within the data. This
is a fairly straightforward approach to teaching machines but decidedly
not how humans learn. We use an approach more akin to what programmers
call unsupervised learning in which the agent experiments on their own
to determine how different courses of action affect their goals. This
type of learning if far more time consuming than supervised learning
because it involves experimentation. The folks at DeepMind are primarily
concerned with unsupervised learning
because it holds the best hope for creating AI with human-like general
intelligence. So while it remains uncertain whether androids will one
day dream of electric sheep, given the social role that we continue to
envision for robots, it does seem increasingly likely that AIs
could soon dream of socially awkward situations like showing up to
school naked. Now exactly what a naked robot might look like is open to
speculation, though Matt McMullen and his teammates at RealDolls seems to have a few vivid ideas of thier own.