Learning in Embedded Systems

Overview

Learning to perform complex action strategies is an important problem in the fields of artificial intelligence, robotics, and machine learning. Filled with interesting new experimental results, Learning in Embedded Systems explores algorithms that learn efficiently from trial-and error experience with an external world. It is the first detailed exploration of the problem of learning action strategies in the context of designing embedded systems that adapt their behavior to a complex, changing environment; such systems include mobile robots, factory process controllers, and long-term software databases.

Kaelbling investigates a rapidly expanding branch of machine learning known as reinforcement learning, including the important problems of controlled exploration of the environment, learning in highly complex environments, and learning from delayed reward. She reviews past work in this area and presents a number of significant new results. These include the intervalestimation algorithm for exploration, the use of biases to make learning more efficient in complex environments, a generate-and-test algorithm that combines symbolic and statistical processing into a flexible learning method, and some of the first reinforcement-learning experiments with a real robot.

About the Author

Endorsements

“Kaelbling’s book is one of the few in the machine learning field that will be regarded as a landmark.”
—Nils J. Nilsson, Kumagai Professor of Engineering, Stanford University

“Learning in Embedded Systems represents he first major attempt at a discussion of the problem of learning action maps.”
—Pattie Maes, Assistant Professor, Media, Arts and Sciences, MIT

“This is likely to become a foundational, problem-establishing book in the rapidly growing area of cognitive science. It includes significant new results, is self-contained and scholarly, and includes excellent references and coverage of related work.”
—Richard S. Sutton, Principal Member of Technical Staff, GTE Laboratories, Inc.