Abstract

Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robot's activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology
Keywords: active learning, autonomy, behavior, complexity, curiosity, sensorimotor development, cognitive development, developmental trajectory, epigenetic robotics, intrinsic motivation, learning, reinforcement learning, values.

Downloads

BibTeX entry

@ARTICLE { philipona:04b,AUTHOR="Philipona, D. and O Regan, J.K. and Nadal, J.-P. and Coenen, O. J.-M. D.",JOURNAL="Advances in Neural Information Processing Systems",TITLE="Perception of the structure of the physical world using multimodal unknown sensors and effectors",VOLUME="16",YEAR="2004",}

Abstract

Color perception is a scathing expositor of the very different points of view that exist about perception.
Besides the philosophical debate about phenomenal experience, colors raise fundamental questions about
the contribution of innate and acquired knowledge at the perceptual level, and about the respective weight of
neuronal and environmental constraints in learning. Colors provide a specially incisive testbed because they
show, in addition to these difficult questions, that the identification of the object of perception is not always
a simple matter: psychophysical experiments indeed support the idea that color perception is concerned
with the reflecting properties of surfaces[1], rather than with light per se as it is often assumed.
The case of color perception recalls an obvious point: before addressing the way neuronal adaptation
takes place in living organisms (be it at the phylogenetical or ontogenetical scale), it is mandatory to question
what they adapt to. This is not a simple question because the nervous system can rely to such an extent
on indirect cues (spectral composition of light) to estimate the object of perception (surface reflectance) that the presentation of these cues elicits a sensation without the presence of the actual object of perception.
It is thus difficult to distinguish, practically and conceptually, what is cue and what is object of perception.
Further, to make things even more complicated, it is problematic to satisfy oneself with an understanding
of the physical object of perception without taking into account the under-determinacy of the sensorimotor
system. For instance, it is obvious that reflecting properties of surfaces concerned with lights outside of
the visible spectrum are irrelevant for color perception. But just as important is the fact that there are
much less obvious aspects within the so-called visible spectrum that are irrelevant as a result of the few
photopigments that biological organisms possess.
We will show how to interpret the physical notion of reflectance in a biological way, so that it involves
only those aspects of the light/surface interaction that are relevant with respect to a given set of molecular
photopigments. From this it will become apparent that surfaces exhibit categorical differences in the way
they modify those aspects of light's spectral composition that impact the photopigments of that set. With
the three photopigment kinds usually assumed for the human visual system, this biological interpretation
predicts eight special colors of three different kinds, corresponding to white/black, red/green/blue and yellow/
purple/cyan. These differences will nally be shown to correspond to differences in the sensorimotor
contingency[2] that the organism engages in when visually exploring colored surfaces.
References
[1] D. H. Brainard. Color constancy. The Visual Neurosciences, 2003.
[2] J. K. O'Regan and A. No¨e. A sensorimotor account of vision and visual consciousness. Behavioral and Brain
Sciences, 24(5), 2001.

BibTeX entry

@INPROCEEDINGS { philipona:04e,ADDRESS="Salk Institute, La Jolla, CA, USA",AUTHOR="David Philipona and Kevin O'Regan and Coenen, Olivier J.-M. D.",BOOKTITLE="Third International Conference on Development and Learning: Developing social brains",MONTH="October",TITLE="Colors and sensorimotor theory, or what do we really perceive?",YEAR="2004",}

Abstract

Visual attention, working memory, reinforcement learning and motor control subsystems are integrated to form a platform for future investigations of the planning and control of complex movements within autonomous robots. The elements of the demonstration are as follows. The ventral pathway, prefrontal areas and frontal eye fields provide an object search, selection and recognition system. A feed-forward motor controller maps desired hand positions in visual coordinates to the pressure ratios required to control a two joint arm powered by two pairs of McKibben air muscles. The combined system is demonstrated on an object recognition and reaching task. Adding a simple basal ganglia model generates sequences of reaching movements to perform a block-copying task. A cerebellum generates sequences of motor commands and the sensory consequences of actions to perform more sophisticated movements as demonstrated in learned throwing and tracking tasks and to facilitate planning at a future stage. Real-time performance of the cerebellar subsystem is achieved through the use of efficient event driven algorithms in software and FPGA-based implementations in hardware of simple conductance-based spiking neurons. The demonstration represents the work of a large and diverse group of researchers with an equally diverse range of research objectives. The different elements have been integrated using the Network Model Interface, a component wrapper which abstracts the interaction between a generic component and a generic framework, and which enables the different components to be investigated and combined within different frameworks and contexts.

Abstract

The motor control system apparently satisfy two sets of data at the
antipode of each other. On one hand, patients that suffer from sensory neuropath
y and are deprived from proprioceptive feedback through the spinal cord are
still able to achieve complex movements using external feedback, such
as vision. On the other hand, animals that underwent spinal cord
transection maintain the ability to execute complex movements, such as
walking on a treadmill. Hence, the first suggests that spinal feedback
is not necessary for motor control, whereas the second demonstrates
that complex motor programs are precisely located within the spinal
cord. We present a control model that attemps to reconcile these two
extremes.
The model is an adaptive feedback control system where the feedback
loop (a spinal stretch reflex model) is driven by the final and
desired muscle lengths. These are computed from desired final
movements of the limbs and provided by higher (than the spinal cord)
brain areas. A learned and adaptive mapping is used to
achieve the correspondance between muscle lengths and limb positions
relative to the body. Feedback gains in the model are adapted
according to the movement context. This ``gain scheduling'' could be
achieved by the higher brain areas by modulating the sensitivity of
the muscle spindle afferents. Finally, another lookup table is used to
store the feedback gains according to the context and type of
movements to perform. We suggest that storage and context modulation
for the parameters may be a role of the cerebellum.
A benchmark reaching movement task is considered to compare results
obtained by our model with results obtained by a common motor control model
put forward in the literature, an inverse model
controller. Simulations are performed with a model of a six-muscle
human arm attached to a robot manipulendum. Similar results are
obtained with both models and are consistent with human behavioral
data. The special case in which the spinal proprioceptive feedback is
abolished (no muscle spindle afferent) is also considered. With this
structure, simulations suggests that reaching movements are still
possible.

Abstract

Human sound systems are invariably phone-
mically coded. Furthermore, phoneme invento-
ries follow very particular tendancies. To ex-
plain these phenomena, there existed so far three
kinds of approaches : Chomskyan"/cognitive
innatism, morpho-perceptual innatism and the
more recent approach of language as a com-
plex cultural system which adapts under the pres-
sure of eÆcient communication". The two first
approaches are clearly not satisfying, while the
third, even if much more convincing, makes a
lot of speculative assumptions and did not really
bring answers to the question of phonemic cod-
ing. We propose here a new hypothesis based
on a low-level model of sensory-motor interac-
tions. We show that certain very simple and non
language-specific neural devices allow a popula-
tion of agents to build signalling systems without
any functional pressure. Moreover, these systems
are phonemically coded. Using a realistic vowel
articulatory synthesizer, we show that the inven-
tories of vowels have striking similarities with hu-
man vowel systems.