The idea of Neodroid is to create a reality-ready robot brain in virtual reality. We specifically focus
on creating a robot brain capable of humanoid visual-motor ability. Visual-motor ability is the
integration between visual perception and motor skills. More specifically, it is the ability to perform
constructive tasks integrating both visual perception and motor skills. The motivation behind
Neodroid is to enable robots to assist humans in performing such tasks.

Humans have the only known visual-motor function that we truly understand through experience.
Even so, we have not been able to reproduce human-level visual-motor function in a robot, despite
the super-human non-visual motor skills in robots. Humans learn through a relatively slow process
that unfolds over years, from the baby stage, through childhood and into adulthood. In Neodroid,
we seek to develop humanoid visual-motor function in an accelerated learning environment, so that
a robot can learn in days or weeks what may take a human years to learn.

The perfect accelerated learning environment is virtual reality (VR), where time can be sped up, and
robots can move and learn faster and safer than in reality. The vision behind Neodroid is to teach
robots how to perform visual-motor tasks, by training their brains in virtual reality. This is similar
in philosophy to how Neo in the movie “The Matrix” learnt kung fu over the course of several hours
within a simulator – in just a few seconds in the real world. Similarly, in Neodroid we will create a
robot brain by training it – using deep learning – in virtual reality, much faster and safer than
possible in the real world.

The essence of Neodroid is shown in Figure 1. The ‘baby’ phase is shown to the left in the picture,
and involves growing the basic visual-motor ability of the brain in virtual reality, so that it can
represent the connections between visual perception and motion. Similar to the baby phase in
humans, where the baby brain transitions from a blank brain – tabula rasa – to a brain capable of
representing the basic structures of reality. There is no human teacher present in the baby phase – in
which environment-assisted learning takes place. Environment-assisted learning implies learning
purely by observation of the environment. In Figure 1 this is illustrated by the virtual robot just
observing images of the falling strawberries together with environment-provided information on the
underlying 3D position and pose of each strawberry. The ‘school’ phase follows, in which a human
enters virtual reality and demonstrates the visual-motor associations and motion patterns that define
different visual-motor tasks. In the school phase, the brain structure develops further and is capable
of representing complex spatio-temporal sequences of visual-motor response patterns. After the
school phase, the brain is transferred from the virtual to the real robot, so learnt visual-motor skills
can be applied in the real world.