Laboratory of Computational Embodied Neuroscience

LOCEN mission is twofold: (a) using computational and robotic models to investigate how brain acquires behaviour, in a cumulative fashion, by interacting with the environment through the body; (b) integrating the knowledge so acquired with state-of-the-art machine learning techniques to build autonomous humanoid robots capable of learning multiple skills in an open-ended fashion.

Additional info

Profile

LOCEN in brief

LOCEN history and overview.

LOCEN, a research group of ISTC-CNR, was founded in 2006 by Gianluca Baldassarre. Before its formation, various members of LOCEN worked in the field of Artificial Life (in particular with Domenico Parisi and Stefano Nolfi): hence the importance given to evolution, adaptation, the function of brain and cognition, embodiment/robots, the interaction of agents with the environment and internal body, and complex systems.

LOCEN has specialised its research in two directions: (a) Autonomous and Developmental Robotics: here the objective is to create autonomous robots capable of open-ended learning based on intrinsic motivations and goals; (b) Computational Embodied Neuroscience models: here the objective is to understand how brain acquires and produces behaviour, and also how it can be affected by impaiments and diseses.

LOCEN is currently formed by 6 Researchers, 4 Research Fellows, and various changing master students and visitors.

The group uses these approaches to investigate two distinct but related sets of topics and problems.

The two methods and the two sets of topics are now illustrated in detail.

LOCEN autonomous robotics

LOCEN autonomous robotics: method

The research agenda and the research bio-inspired approach of the group

The group has a research agenda that aims to build autonomous cumulative learning robots. This are robots that are capable of autonomously acquiring a number of skills in autonomous fashion, without human intervention. Although the group fully recognise the great importance of social mechanisms for the acquisition of complex behaviours like those acquired by primates (e.g., social enhancement, joint attention, imitation, language, teaching, etc.), the group focusses its research on individual learning processes so to have a higher impact in this study. In this respect, the ``holy grail'' of LOCEN is to arrive to build robots that, like children, are capable of acquiring an increasingly sophisticated repertoire of sensorimotor and cognitive skills, from simple to complex ones, in a cumulative fashion, without external intervention. For this reason, we follow the developmental robotics approach to build its robots.

In this respect, the group believes that fundamental breakthroughs in specific fields of robotics and machine learning will come from the study of biological systems. The reason is that organisms are the result of a run of the ``gigantic'' genetic algorithm represented by natural evolution: such algorithm has found solutions to some problems that engineering approaches will not beat for long. The idea is so to reverse-engineer brain and behaviour of real animals to propose radically new ideas to autonomous robotics and machine learning, that might synergies with those discovered with more traditionalapproaches.

LOCEN autonomous robotics: topics

The topics investigated by the group are all directed to support its ``mission'' of building autonomous, cumulative learning robots. We believe that four critical challenges need to be overcome to accomplish this goal:

Dynamic behaviour, compliant hardware and muscles

All that ultimately counts, for both robots and animals, is action. Action means a change of the sensorimotor body in space that in turn can change the environment (e.g., to manipulate objects) and the body-environment relation (e.g., to look at something, or to change position in space). Behaviour emerges from the dynamic interplay between the robot's action, that changes the body-environment relation, and the close-loop physical and perceptive feedback flow that results from this. Part of the extraordinary capabilities that animals display in these interactions rely upon the hardware of their body, in particular their compliant skeleto-muscular system. For this reason, one topic of interest of the group is the challengs and opportunities offered by robots endowed with a biomorphic compliant hardware (e.g., bio-materials, biologically-plausible structure, pneumatic actuators).

Active vision, proprioception and abstraction

The sensorimotor flow brings to robots and organisms a huge amount of information. The group focuses on proprioception, touch, and vision. Proprioception is used to perform dynamic motor behaviour. Vision is instead used to guide behaviour at a higher level, e.g. to visually guide reaching and grasping, or to decide with which object to interact. We see perception not as a passive source of information, but rather as an active process of search of relevant information and avoidance of non-useful information given the goal pursued by the robot/organism. At both the perceptive and motor level, behaviour needs abstractions. For example, when an object is seen it is useful to abstract a number of distinct information elements used by different downstream processes of the controller, e.g. its location, size, identity, etc. Active vision and abstraction processes are so an important investigation topic of the group. ouch are used as a fundamental element to implement sensorimotor skills at the low level.

Extrinsic and intrinsic motivations, and learning

As the group is interested in autonomous cumulative learning, he is profoundly interested on the mechanisms that can guide learning in an open-ended fashion without external intervention. Motivations are a key element for this as they can: (a) drive the robots to perform different behaviours in different conditions; (b) generate the learning signals needed to guide their learning. Two types of motivations are studied by the group. The first are extrinsic motivations, i.e. motivations related to the task assigned to the robot (what is the best reward function function?). The second are intrinsic motivations. These are a rather novel concept in autonomous robotics and the group is keenly investigating them. Intrinsic motivations drive behaviour and generate learning signals on the basis of the fact that the controller is indeed acquiring knowledge with a high rate, e.g. is forming good abstractions, is learning to predict well, or is acquiring the sensorimotor competence to accomplish a goal. Intrinsic motivations are fundamental for autonomous robots as allows them to acquired knowledge and skills in the absence of tasks from the user, so that when some tasks are assigned to them they can exploite the knowledge acquired autonomously to readily solve them. The pivot of all these processes is of course learning, so the group keenly investigates all forms of learning useful for robots, from associative to unsupervised learning, from supervised to reinforcement learning (the latter is central in several models of the group).

A video showing a humanoid robot (iCub) that explores a `mechatronic board' under the drive of intrinsic motivations, and autonomously discovers and learns that pressing some buttons turns on some lights. In a later stage, the robot exploits the acquired action-outcome contingencies to accomplish useful purposes. For more detailed videos see here and here: these videos are explained in the web-pages here and here; the system is explained in detail in the article here. Work carried out within the project IM-CLeVeR.

Hierarchical architectures

The cumulative acquisition of skills and cognitive capabilities requires that the newly acquired knowledge: (a) is stored without destroying previously acquired knowledge; (b) supports the acquisition of further knowledge. The first goal is achieved through hierarchical architectures that avoid that the acquired different pieces of knowledge interfere with each other. The second goal is accomplished by investigating how the acquired knowledge can be transferred to new tasks to be solved (e.g., as in transfer reinforcement learning). Because of these two goals, the group is keenly interested in developing hierarchical reinforcement learning architectures.

Video of a humanoid robot (iCub) learning to throw a ball to a target based on a hierarchical reinforcement learning system with sophisticated generalisation capabilities (generation of new dynamic movement primitives on the fly based on the similarity of the new goal with respect to previously acquired goals). This work was carried out in collaboration with Bruno Castro da Silva and Andrew Barto, from the University of Amherst Massachusetts.

LOCEN brain and behaviour

LOCEN brain and behaviour: method

Computational Embodied Neuroscience

The group has developed an original research method to study brain and behaviour, rooted on system-level computational neuroscience and artificial life, named Computational Embodied Neuroscience (CEN).

Differently from other computational neuroscience approaches, CEN aims to understand the brain with a ``top-down approach'' starting from behaviour and function. The key idea behind this is that the brain evolved to allow animals to act so as to improve their survival and reproductive chances. So to fully understand how brain works, we need to understand not only its mechanisms (anatomy and physiology, the common focus of neuroscience) but also its functions, i.e. ``what it is for''. This is why the group is keenly interested in linking the knowledge produced not only by neuroscience but also by psychology and psychobiology.

From system-level to detailed models (multi-scale models)

Another consequence of the goal of CEN of studying whole systems capable of acting is the tendency to build system-level models, reproducing the macro-architecture of various cortical and sub-cortical brain areas that underlie the target behaviours. This is in fact needed if one wants to understand how a certain area of brain works not studying it in isolation, but how its inner mechanisms play a certain function within a whole system. Of course, often the aspect of the system which are not under focus are represented in an abstract fashion, but they are nevertheless there. After sedimenting knowledge at the system-level, the group usually starts to refine the micro-architecture and functioning of the various components of the model (usually focussing on a subset of them). This leads to have multi-scale models that encompass not only the system-level but also the meso-level (e.g., different nuclei of basal ganglia) and micro-level (e.g., the canonical architecture of cortex) of brain. For these overall purposes, our models are usually based on firing rate neurons and leaky neurons, and only recently on spiking neurons (integrate-and-fire neurons). Recently, the group has started to build probabilistic graphical models whose functioning is implmemented on the basis of Bayesian inferences. These models have the advantage of facilitating system-level modelling at a high (often functional) level before moving to more detailed neural models, and also of offering the possibility to directly evaluate the goodness of models against empirical data, for example from brain imaging. We are also interested in probabilistic interpretations of brain processing (from which our recent interest for spiking neurons).

Aiming to build cumulative models

The hallmark of science is cumulativity. Too often different computational models are build to interpret different experiments. Instead, CEN aims to build models that allow the explanation of an increasing number of specific brain and behavioural data. This allows the isolation of general principles, the integrated theoretical systematisations of whole classes of phenomena, and so to help psychology and neuroscience to overcome the polverisation of results and views that they often encounter for their need of focussing. Integrated theories and models allow the production of detailed hypotheses that fill in the knowledge gaps of psychology and neuroscience and produce specific empirical predictions that can be tested in new empirical experiments.

The importance of a close dialogue between models and empirical data

Differently from other computational approaches to the study of brain and behaviour, CEN stresses the importance of having a tight, continuous dialogue with specific empirical data from psychology and neuroscience. The idea is that the understanding of brain and behaviour should proceed as any good science, namely it should rest on two pillars: (a) the theoretical understanding of the investigated phenomena, based on formal computational models; (b) the empirical investigation of such phenomena to select the best theories, models, and predictions. In this respect, we see computational modelling as a powerful theoretical means that should guide empirical research on brain and behaviour. The ultimate proof that computational modelling of brain and behaviour has successfully accomplished its mission is its capacity to change the daily research of the empirical neuroscientist and psychologist, and to publish papers in top journals of neuroscience and psychology.

The importance of embodiment: sensorimotor and visceral

We believe that brain generates behaviour by dynamically interacting with the environment through sensors and actuators in a circular fashion (embodiment). Sensors furnish a rich, redundant and noisy amount of information to organisms. Actuators are in turn noisy, redundant, compliant. Before facing high-level cognition problems, brain has to solve the problems posed by such input/output information channels (and also exploit the opportunities they offer) . The resulting computations might be radically different from those that would stem from, say, a clean, symbolic type of input/output information. For these reasons, we think that good models of brain and behaviour should function in simulated or real robotic systems that have the same sensors and actuators as the investigated animals. This poses strong challenges to models, especially because neuroscientists and psychologists often overlook them and require us to focus ``on their problems'', concerning higher-level aspects of cognition. For this latter reason, the more biologically constrained models produced by the group so far often use localistic representations, abstract input/output information codes, simplified environments. However, the group is fully aware of the importance of scaling up models to more realistic input/output and environmental conditions, so we try our best to incorporate in the models critical elements of a true embodiement, e.g. the sensorimotor loop and the test with simulated/real robots. Aside this, the group gives also a lot of important to a second type of ``embodiment'', most of the times neglected but as important as ``sensorimotor embodiment''. This might be called ``visceral embodiment'' and refers to the key relation that the brain has with the visceral body and its homeostatic regulations. These regulations are at the basis of extrinsic motivations and the subjective value (biologically saliency) that organisms assign to objects and experiences.

The importance of learning

We believe that for a large part the brain structure is as it is because it has not only to express behaviour but also to learn it. For this reason, most of our models aim not only to reproduce target behaviours, and the neural machinery to do so, but also the learning processes that lead to its acquisition with experience, and hence the physiological processes underlying this. For this reason we are keenly interested in studying all forms of biologically plausible learning: Hebbian learning, differential Hebbian learning, competitive self-organised learning, and reinforcement learning, and goal-based learning.

LOCEN brain and behaviour: topics

Given the interest of the group in understanding how brain and behaviour supports cumulative learning in organisms, our research is focussed on the following topics:

The bio-contrained models developed by the group focus on two types of perception: proprioception and vision. Often we use proprioception in our models to guide low-level behaviour, but we do not study it per se. We instead are very interesting in studying vision, and especially active vision. The reason is that vision is the primary information source for primates and has a paramount role in guiding action (via the brain dorsal pathway, involving parietal and premotor cortex) and to support higher-level cognition, such as decision making and planning (via the ventral pathway, involving inferotemporal cortex and prefrontal cortex). We study vision not as a passive source of information but rather as an active one. In particular, we are interested in studying how overt attention, and the high resolution of the fovea with respect to peripheral vision, can actively search and gain information in the environment, and ignore irrelevant one, on the basis of the animal's goals. We are thus interested in bottom-up attentional processes, that drive eye gaze on most informative parts of the environment, and in top-down attentional processes, that drive the eye to collect information based on goals; and we are of course interested in their rich interplay. We also believe that attention is pivotal for the rest of behaviour and indeed there is a strong coupling between vision and arm/hand manipulation actions, with attention representing a powerful guidance for controlling such actions via the selection of suitable inputs (``eyes lead, arms execute'').

Behaviour needs to be driven. Learning needs to be guided. Extrinsic motivations are expressed by parts of brain (amygdala, nucleus accumbens, dopaminergic and other neuromoduatory systems) at the interface between visceral body and cognitive processes. We are interested in studying how extrinsic motivations drive behaviour, directing it to specific activities, or how they generate learning signals that guide learning processes. In this respect, we are very interested in investigating how areas such as the amygdala can perform Pavlovian associations that allow triggering important internal reactions (e.g., for the internal regulation of visceral body and the neuromodulation of brain) and external behavioural reactions (feeding, approaching, orienting) in correspondence to biologically salient stimuli (e.g., food, water, sex) or stimuli anticipating them (conditioned stimuli). Also, we are very interested in understanding instrumental behaviour, i.e. the processes that allow organisms to learn to trigger (learned, instrumental) behaviours, when particular conditions are present, if this leads to rewards (S-R behaviour).

Aside extrinsic motivations, we are interested in studying intrinsic motivations, i.e. the motivations at the core of the performance and acquisition of actions ``for their own sake'', i.e. not for the achievement of results that directly increases biological fitness (e.g., food or money). Intrinsic motivation systems have evolved as they drive exploration and learning in the absence of extrinsic outcomes, and have the function of leading animals to acquire knowledge and skills that will be readily usable in the future when such extrinsic outcomes become available.

For example, intrinsic motivations are maximally apparent in children at play: in the absence of homoeostatic drives, children engage in ludic behaviours driven by curiosity, novelty, surprise, and changes in the environment, in general all experiences that cause an improvement of their knowledge and skills. Neuroscience is unrevealing some of the brain mechanisms behind these processes, e.g. the capacity of hippocampus to cause dopamine production (learning signals) when a novel object is perceived, or the capacity of superior colliculus to produce phasic dopamine when the world change, or the capacity of frontal cortex to cause noradrenaline production when predictions are violated.

Organisms' cumulative learning of sensorimotor behaviour and higher-level cognition requires a hierarchical soft-modular brain architecture that links motor behaviour to perception at different levels of abstractions and coupling. This architecture is organised in at least three levels, investigated by the group with system-level models. (1) At the lowest level, the close-loop between somatosensory cortex and primary motor cortex (forming a loop with sensorimotor basal ganglia involving putamen/globus pallidum) and involving cerebelllum, implements the dynamic production of movements (e.g., a reach, a grasp); (2) the dorsal brain pathway, encompassing visual cortex, parietal cortex (encoding affordances), and premotor cortex (encoding repertoires of actions) (which forms a loop with sensorimotor basal ganglia involving putamen-caudatum/pallidum-subtantia nigra reticulata), implement the on-line control of action (e.g., to guide a reach to an object, or to shape the hand to grasp it); (3) the ventral brain pathway, encompassing the visual cortex, the inferotemporal cortex (for object recognition), and the prefrontal cortex (for working memory and multimodal sensory integration) (which forms a loop with associative basal ganglia involving caudatum/pallidum), implements high-level decision making and executive control of behaviour; (4) the cortex communicating with sub-cortical limbic structures, such as amygdala and hippocampus (which forms a loop with limbic basal ganglia involving nucleus accumbens), processes value and encodes biologically salient action-outcomes and goals (e.g., food). The group studies this complex hierarchical system by usually focussing on different parts of it but always keeping in mind the overall architecture.

Starting form sensorimotor behaviour, the group is now gaining knowledge and skills for the investigation of higher-level cognition, in particular in relation to goal-directed behaviour and decision making (levels 3 and 4 of the framework of the previous point). ``Goals'' are now becoming a critical concept for the group for the pivotal role they play in cognition and, in particular, in autonomous cumulative learning. A goal is an internal representation of a future world state that is activated internally and drives action (and learning) for its accomplishment. The current investigation of ``goal-directed behaviour'' and decision making, strongly supported by theory-driven model-free and models-based reinforcement learning, is one of the hottest fields of investigation of cognitive science. Goals also play a key role in autonomous cumulative learning: a critical function that intrinsic motivations play is the self-generation of goals. If you look with attention a little child at play, you will realise how her/his autonomous exploration and learning is strongly guided by an incessant autonomous setting, pursuit, accomplishment, monitoring, and switch of goals. Goal-based processes, greatly enhanced by the uniquely-developed human prefrontal cortex, focus attention, inform on success, drive learning, and organise action in ways that render autonomous learning ``explosive'', i.e. powerful, omni-directional and open-ended.

LOCEN interactive devices

Since 2015 the group started collaborating with some rehabilitation centers to realise interactive devices for therapeutic use. This collaboration led to the implementation of "+me", a working experimental prototype addressed to the therapy of Autism Spectrum Disorders. The device is currently in clinical trial. More info on www.plusme.it

Concluding remarks

The presence in the same group of different perspectives and methods (robotics, modelling, neuroscience and psychology) on the same topics give the group a number of advantages:

An approach to problems of autonomous robotics and cognitive-science that is profoundly interdisciplinary

The capacity to coordinate and play a key role in large robotics/cognitive-science projects involving different approaches, themes, and teams

An ``eagle-eye view'' on robotics and cognitive-science issues that allows the group to see phenomena, problems, and solutions overlooked by other more focussed, but possible near-sighted, views and approaches.

Research

OVERVIEW AND Focus Web-Pages on LOCEN Research Threads

Table: below we report a table listing the main research threads of LOCEN, organised by two main themes:(1) IMOD: Intrinsic Motivations and Open-ended Development in robots and animals(2) GAH: Goals, Actions, and Hierarchies in robots and animalsThe research threads are also organised by their main focus (but most threads are highly interdisciplinary and involve more approaches):(a) Robots;(b) Behaviour;(c) Brain.

Overview: below the table we overview each research thread, and we give the link to a focus web-page containing more explanations and material (we are working on them to add detailed explanations, pictures, videos, list relevant publications, etc. ).

ROBOTS: INTRINSIC MOTIVATIONS AND OPEN-ENDED DEVELOPMENT

Focus page:Further details on this research threadProject dedicated page:IM-CLeVeRAuthors: Vieri Santucci, Daniele Caligiore, Kristsana Seepanomwan, Marco Mirolli, Gianluca Baldassarre (but the project involved most LOCEN)Topic and its relevance. This research thread is relative to the robotic works carried out within the IM-CLeVeR EU funded project. Developing robots able to autonomously discover, select, and solve multiple new tasks in a cumulative open-endeed fashion is an important issue for autonomous robotics. It becomes even crucial if we want to build robots capable of solving multiple problems in real environments posing challenges that are unknown at design time. There are two key `ingredients' necessary to build these kind of robots. The first are intrinsic motivations (IMs): these can drive autonomous learning of robots in an open-ended fashion in the absence of tasks assigned to the robots by the users. The second are hierarchical architectures: these are needed to store multiple skills, drive their acquistion with IMs, learn goals related to skills, and form complex skills based on simpler skills.

Active vision and open-ended learning

Focus page: Further details on this research threadAuthors: Dimitri Ognibene, Valerio Sperati, Rodolfo Marraffa, Gianluca BaldassarreTopic and its relevance. This project (initially funded by the EU project MindRACES and later by the EU project IM-CLeVeR) is on LOCEN's approach to vision called `Ecological Active vision - EAV'. EAV is grounded on the `active vision' approach, based on an actively-moved small fovea plus a low-resolution periphery, augmented with four principles: (a) a strong coupling of bottom-up and top-down attention processes; (b) the use of reinforcement-learning to acquire top-down attention skills; (c) the use of attention and vision to support pragmatic action (e.g., reaching and grasping) rather than vision per-se, in particular a strong spatial coupling between attention and manipulation actions; (d) the use of a novel Potential Action Memory component to collect information on the best places to visit with the fovea. Lately we have linked EAV with intrinsic motivations (IMs), in particular IM related to the perception of movement in the world and agency (i.e., the agent's perception of the capacity to cause movement in the world with own actions).

Focus page:Further details on this research threadAuthors:Daniele Caligiore, Paolo Tommasino, Annalisa Ciancio, Valentina Meola, Gianluca BaldassarreTopic and its relevance:Building architectures that allow robots to learn multiple sensorimotor skills, possibly transferring knowledge between them, is a central open challenge for autonomous robotics. In particular, it is paramount to produce autonomous cumulative learning robots. This is also important to suggest possible architectures and processes through which brain solves the same problems.

Echo-state networks and dynamic motor behaviour: robots and brain

Focus page:Further details on this research threadAuthors: Francesco Mannella, Gianluca BaldassarreTopic and its relevance. This research thread, started recently, concerns the use of echo-state networks and the modulation of their dynamics. Echo-state networks are an important class of neural networks belonging to the family of models called ``dynamic reservoires''. Echo-state networks have very interesting and powerful computational properties that make them suitable to learn and produce complex motor behaviours relevant for both robotics and for the study of motor behavoiur produced by brain. On the robotic side, the importance of this resides in the fact that the learning and production of sophisticated discrete and rhythmic movements is a pivotal building block of autonomous robotics architectures. In this respect, our approach is usable to face problems for which the autonomous robotics litearture uses devices such as Dynamic Movement Primitives (DMPs). The advantages of our approach with respect to DMPs is expected to be in terms of sophistication and flexibility of the movements producible with echo-state networks. The work is also important for brain modelling, in particular to model cortex viewed as a dynamical system whose dynamics is regulated by basal ganglia. The importance of this resides in the fact that the basal-ganglia and cortex form segregated loops that are a fundamental building module underlying multiple brain processes, from associative sensory processing, to motor behaviour, thinking, plannig, and reasoning.

Collective navigation robotics

Focus page:Further details on this research threadExternal dedicated web-page:Collective navigating robotsAuthors: Gianluca Baldassarre, Domenico Parisi, Stefano NolfiTopic and its relevance. This is research thread, now terminated, was conducted within the EU funded project Swarm-bots. Collective robotics involves the use of multiple robots to carry out tasks that could not be carried out by single robots alone. For some tasks, the simplicity of single robots in terms of sensors, actuators, and communication capabilities can give robustness and low-cost to the whole ``swarm'' of robots. In this cases, the coordination between robots, needed to carry out a common task in cooperation, can rely on distributed (vs. centralised/hierarchical) coordination and communication mechanisms typically exploited by social insects (e.g., ants and bees), for example stigmergy (what one robot does with its body and in the environment is directly exploited by the others for coordination).

BEHAVIOUR: INTRINSIC MOTIVATIONS AND OPEN-ENDED DEVELOPMENT

Intrinsic motivations: theory and empirical experiments

Focus page:Further details on this research threadAuthors: Vieri Santucci, Daniele Caligiore, Magda Mustile, Marco Mirolli, Gianluca BaldassarreTopic and its relevance. Intrinsic motivations (IMs) are related to curisity, exploration, the interest for novel objects and surprising evens, and the drive to learn motor skill. IMs operate in the absence of a direct biological pressure and feedback (as in the case of extrinsic motivations, i.e. the classic motivations related to homeostatic regulations and survival). IMs are a fundametnal topic of investigation as they play a key role in human well being, art, science, and technology. They are also important for autonomous robotics as they allow the construction of cumulative learning robots. We are elaborating a general theory on intrinsic motivations. This offer predictions, testable in empirical experiments (examples are reported in this research thread), and mechanisms for computational models (see other research threads).

Authors: Beste Ozcan, Valerio Sperati, Tania Moretta, Laura Romano, Daniele Caligiore, Gianluca Baldassarre (ISTC), Simone Scaffaro, Alessandro Medda (INI Villa Dante)Topic and its relevance. Autism Spectrum Disorders -ASD- are a set of neurodevelopmental conditions characterised by the impairment, in varying degrees, of three basic areas for the psychic development of children: the social interaction; the communication(both verbal and not); the repertoire of activities and interests. This project is direced to realise an interactive wearable device that can potentially support and motivate the development of basic social skills.

BEHAVIOUR: GOALS, ACTIONS, HIERARCHIES

Children's development of reaching skills: an embodied model (iCub simulator)

Focus page:Further details on this research threadAuthors: Daniele Caligiore, Domenico Parisi, Gianluca BaldassarreTopic and its relevance: Reaching, i.e. the capacity to get own hands in contact with objects in the environment, is a fundamental motor skill for primates and humans, at the basis of their capacity to interact with, manipulate, and change the world at own benefit. Here we use computational models to understand the mechanisms underlying learning and development of such reaching skill. The study of reaching is supported by the availability of a wealth of empirical data against which models can be tested. The model is mainly used to: (a) reproducedata obtained with real children; (b) explain brain/body mechanisms behind such data; (c) produce testable predictions.

BRAIN: INTRINSIC MOTIVATIONS AND OPEN-ENDED DEVELOPMENT

IM-CLeVeR EU project: basal ganglia-cortex hierarchies in brain

Focus page:Further details on this research threadAuthors: Francesco Mannella, Vincenzo Fiore, Valerio Sperati, Marco Mirolli, Gianluca BaldassarreTopic and its relevance. We describe here research works, funded by the EU project IM-CLeVeR, directed to investigate what is the architecture and mechanisms of brain that allow primates (e.g., monkeys and children) to learn multiple skills in an cumulative fashion on the basis of intrinsic motivations. The overall architecture of brain relevant for this topic is the same as the one illustrated above in the research thread on goal-directed behaviour and habits, with the addiction of further structures important for intrinsic motivations such as the superior colliculus (important to detect changes in the environmetn caused by the organism), hippocampus (important to detect novel patterns and events), and prefrontal cortex (important to detect violation of expectations).

BRAIN: GOALS, ACTIONS, HIERARCHIES

Brain cortico-cortical pathways and hierarchies

Focus page:Further details on this research threadAuthors: Daniele Caligiore, Anna Borghi, Domenico Parisi, Gianluca BaldassarreTopic and its relevance. "Embodied cognition", postulating that high-level cognition relies on the same brain mechanisms subserving sensorimotor behaviour, is a frontedge flourishing research topic of cognitive psychology and cognitive neuroscience (e.g. related to mirror neurons). Here we use computational models and theoretical analysis to propose sufficient hypotheses on the architecture, functioning, and learning processes through which the dorsal and ventral cortical pathways of brain can guide on-line action control (based on affordances and motor programs) and top-down control of them (based on context, internal motivations, and goals).

Models of goal-directed and habitual behaviors

Focus page:Further details on this research threadAuthors: Francesco Mannella, Vincenzo Fiore, Marco Mirolli, Gianluca BaldassarreTopic and its relevance. Organisms have a brain that evolved to produce a behaviour that enhances their survival and reproductive chances. To do this, brain produces body movements (actions) in correspondence to sensations. More sophisticated organisms, as primates, need to learn to perform and appropriately select a large number of actions depending on the environmental conditions and current internal needs. A very complex hierarchical brain architecture underlies these processes. This architecture involves the production of dynamic movementes/actions (somatosensory cortex, primary motor cortex, dorsal basal ganglia, cerebellum), their selection (premotor cortex, dorsal/medial basal ganglia), their selection and sequencing based on the organism's goals (dorsolateral prefrontal cortex, supplementary motor cortex, medial basal ganglia), their selection at a higher level based on the organisms' ultimate motivations and needs (hypothalamus, amygdala, ventral basal ganglia, orbital and ventromedial prefrontal cortex). Goal-directed behaviour involves the triggering of actions on the basis of internal motivations and goals: this typically involve amygdala, ventral/medial basal ganglia, and orbital and ventromedial prefrontal cortex. Habitual behaviour (habits) involve the automatic triggering of actions in the presence of a particular external and internal context: they typically involve dorsal basal ganglia and premotor/primary cortex. Although we have much evidence on these issues, we are still far from having a complete whole picture on these processes. The importance of this research resides in the fact that understanding these processes means understanding a large part of the whole brain functioning.

Pavlovian to instrumental transfer: models, behaviour, brain

Focus page:Further details on this research threadAuthors: Emilio Cartoni, Gianluca BaldassarreTopic and its relevance. The study of intrumental and habitual behaviour (see above) has led us to investigate their interactions by addressing a specific psychobiology experimental paradigm called `Pavlovian Instrumental Transfer' (PIT). This issue is very important as Pavlovian and Instrumental processes are fundamental learning processes underlying adaptive behaviour.

Focus page:Further details on this research threadAuthors: Daniele Caligiore, Francesco Mannella, Gianluca BaldassarreTopic and its relevance.The basal ganglia and cortex forms re-entrant loops through which basal ganglia contribute to select the contents of cortex. Also cerebellum forms re-entrant loops with motor cortex subserving both motor and cognitive functions. Recent evidence has also shown the existence of important cerebellum-basal ganglia anatomical bidirectional links. Basal-ganglia, cortex, and cerebellum thus form a whole system and closely cooperate to implement a large number of brain functions subserving adaptive behaviour. The specific brain mechanisms underlying these functions are only in part known. Computational models can play a key role in understanding them given the highly-dynamical complex-system nature of the basal ganglia-cortex-cerebellum system. The study of this system is also important for understanding and treating neurodegenerative diseases, such as Parkinson caused by the progressive death of dopaminergic neurons. Indeed, the system-level model-based study empowers the tracing of the effects that the Parkinson dopaminergic disregulation causes onto the whole basal ganglia-cortex-cerebellum system, thus helping to understand the multifaceted sympthoms of the diseases expressed in different patients sub-types (e.g., tremor vs. akinetic).

Laboratories

Learning paths for people wishing to work with LOCEN

Below we depict the ''learning paths'' that students and young researchrs should follow to acquire the knowledge and skills needed to work with LOCEN. The different paths depend on the different research interests of the person, covering a specific area of research of LOCEN.

How to get to the ISTC-CNR: directions, maps, hotels

DIRECTIONS

From the airport “Roma Fiumicino”

This is the main airport of Rome (see map below). Follow the indications for the train terminal inside the airport. At the train terminal, buy the tickets at the ticket office for the train “Leonardo Express” to “Roma Termini” central station, where you have to get off (last stop). The ticket can be purchased at the ticket office, at the news agents and at the automatic ticket machines (cost: 11 euros). The train runs from 6.30 am to 11.30 pm, and departs every 30 minutes. Once at Termini Station, you can reach ISTC-CNR on foot (10 minutes, see map below) or take the subway. To take the subway, look for signs of subway B, Rebibbia direction (“Metro B, direzione Rebibbia”; Rome has only two subway lines, A and B.). Buy tickets from the automatic ticket machines (cost: 1 euro). Get off after one stop at subway stop “Castro Pretorio”. ISTC-CNR, a brown historical two-floor building, is round the corner of this subway stop at the beginning of Via S. Martino della Battaglia, at the entrance number 44 (see map below).If you want to spend less money for the train, at Fiumicino airport take the train to “Roma Tiburtina”, where you get off (the trip takes about 45 minutes as this is a local train; this is usually not the last stop). At the train terminal, you can buy the ticket for this train (cost: 4.5 euros). Once at Tiburtina Station (the second biggest station in Rome), take the subway B, Laurentina direction (“Metro B, direzione Laurentina”). Get off at subway stop “Castro Pretorio”. As explained above, ISTC-CNR is round the corner of this subway stop.The taxi from Fiumicino airport to Rome costs about 40 euros (supplements might be asked for luggage, night-time runs and public holidays) and the trip takes approximately 45 minutes.

From the airport “Roma Ciampino”

This is the second airport of Rome, and is very small. It is closer to Rome than Fiumicino airport (see maps below). Get out of the airport terminal and ask the bus drivers standing outside, near the buses you see once out, about a bus that takes you to Anagnina Subway Station (“Stazione della Metropolitana Anagnina”). The subway of Anagnina Station is on the subway line A. Once at Anagnina Station, buy a ticket from the automatic ticket machines or the news agents (cost: 1 euro). At Anagnina Station, take the subway A to Roma Termini Station. Once at Termini Station, you can reach ISTC-CNR on foot (10 minutes away, see map below) or take the subway. Otherwise, switch subway line to subway B following the signs for “Metro B, direzione Rebibbia”. Get off after one stop at subway stop “Castro Pretorio”. ISTC-CNR, a brown historical two-floor building, is round the corner of this subway stop at the beginning of Via S. Martino della Battaglia, at the entrance number 44 (see map below).Taxis from Ciampino airport to ISTC-CNR charge about 30 euros, and take about 20 minutes to get there.

MAPS

Rome has two airports, "Fiumicino" and "Ciampino". ISTC-CNR is situated at the centre of Rome: