Intrinsically Motivated Learning in Natural and Artificial Systems

Intrinsically Motivated Learning in Natural and Artificial SystemsIt has become clear to researchers in robotics that current approaches are yielding systems with limited autonomy and capacity for self-improvement. To learn autonomously and in a cumulative fashion is one of the hallmarks of intelligence, and we know that higher mammals engage in exploratory activities with the environment that are not directed to pursue goals of immediate relevance for the survival and reproduction of the organism but are instead driven by intrinsic motivations such as curiosity, interest in novel stimuli or surprising events, and inter¬est in learning new behaviours. The adaptive value of such intrinsically motivated activities lies in the fact that they allow the cumulative acquisition of knowledge and skills that can be later used to accomplish fitness-enhanc¬ing goals. Intrinsic motivations continue during adulthood, and in humans they underlie lifelong learning, artistic creativity, and scientific discovery, while they are also the basis for processes that strongly affect human wellbeing, such as the sense of competence, self-determination, and self-esteem.This book has two aims: to present state-of-the-art research on intrinsically motivated learning, and to identify the related scientific and technological open challenges and research directions. After explaining the concept of intrinsic motivations, the book introduces a taxonomy of three classes of intrinsic-motivation mechanisms: those based on predictors, on novelty detection, and on competence. The remaining chapters are organized into six parts: the chapters in Part I give general overviews on the concept of intrinsic motivations, their function, and possible mechanisms for implementing them; Parts II, III, and IV focus on the three above-mentioned classes of intrinsic-motivation mechanisms; Part V discusses mechanisms that are complementary to intrinsic motivations; and Part VI introduces tools and experimental frameworks for investigating intrinsic motivations.The contributing authors are among the pioneers carrying out fundamental work on this topic, drawn from related disciplines such as artificial intelligence, robotics, artificial life, evolution, machine learning, developmental psychology, cognitive science, and neuroscience. The book will be of value to graduate students and academic researchers in these domains, and to engineers engaged with the design of autonomous, adaptive robotics.