For complex robots such as humanoids, model-based control is highly beneficial for accurate tracking while keeping negative feedback gains low for compliance. However, in such multi degree-of-freedom lightweight systems, conventional identification of rigid body dynamics models using CAD data and actuator models is inaccurate due to unknown nonlinear robot dynamic effects. An alternative method is data-driven parameter estimation, but significant noise in measured and inferred variables affects it adversely. Moreover, standard estimation procedures may give physically inconsistent results due to unmodeled nonlinearities or insufficiently rich data. This paper addresses these problems, proposing a Bayesian system identification technique for linear or piecewise linear systems. Inspired by Factor Analysis regression, we develop a computationally efficient variational Bayesian regression algorithm that is robust to ill-conditioned data, automatically detects relevant features, and identifies input and output noise. We evaluate our approach on rigid body parameter estimation for various robotic systems, achieving an error of up to three times lower than other state-of-the-art machine learning methods

One of the hallmarks of the performance, versatility, and robustness
of biological motor control is the ability to adapt the impedance of
the overall biomechanical system to different task requirements and
stochastic disturbances. A transfer of this principle to robotics is
desirable, for instance to enable robots to work robustly and safely
in everyday human environments. It is, however, not trivial to derive
variable impedance controllers for practical high degree-of-freedom
(DOF) robotic tasks.
In this contribution, we accomplish such variable impedance control
with the reinforcement learning (RL) algorithm PISq ({f P}olicy
{f I}mprovement with {f P}ath {f I}ntegrals). PISq is a
model-free, sampling based learning method derived from first
principles of stochastic optimal control. The PISq algorithm requires no tuning
of algorithmic parameters besides the exploration noise. The designer
can thus fully focus on cost function design to specify the task. From
the viewpoint of robotics, a particular useful property of PISq is
that it can scale to problems of many DOFs, so that reinforcement learning on real robotic
systems becomes feasible.
We sketch the PISq algorithm and its theoretical properties, and how
it is applied to gain scheduling for variable impedance control.
We evaluate our approach by presenting results on several simulated and real robots.
We consider tasks involving accurate tracking through via-points, and manipulation tasks requiring physical contact with the environment.
In these tasks, the optimal strategy requires both tuning of a reference trajectory emph{and} the impedance of the end-effector.
The results show that we can use path integral based reinforcement learning not only for
planning but also to derive variable gain feedback controllers in
realistic scenarios. Thus, the power of variable impedance control
is made available to a wide variety of robotic systems and practical
applications.

1999

This review will focus on two recent developments in artificial intelligence and neural computation: learning from imitation and the development of humanoid robots. It will be postulated that the study of imitation learning offers a promising route to gain new insights into mechanisms of perceptual motor control that could ultimately lead to the creation of autonomous humanoid robots. This hope is justified because imitation learning channels research efforts towards three important issues: efficient motor learning, the connection between action and perception, and modular motor control in form of movement primitives. In order to make these points, first, a brief review of imitation learning will be given from the view of psychology and neuroscience. In these fields, representations and functional connections between action and perception have been explored that contribute to the understanding of motor acts of other beings. The recent discovery that some areas in the primate brain are active during both movement perception and execution provided a first idea of the possible neural basis of imitation. Secondly, computational approaches to imitation learning will be described, initially from the perspective of traditional AI and robotics, and then with a focus on neural network models and statistical learning research. Parallels and differences between biological and computational approaches to imitation will be highlighted. The review will end with an overview of current projects that actually employ imitation learning for humanoid robots.

While it is generally assumed that complex movements consist of a sequence of simpler units, the quest to define these units of action, or movement primitives, still remains an open question. In this context, two hypotheses of movement segmentation of endpoint trajectories in 3D human drawing movements are re-examined: (1) the stroke-based segmentation hypothesis based on the results that the proportionality coefficient of the 2/3 power law changes discontinuously with each new â??strokeâ?, and (2) the segmentation hypothesis inferred from the observation of piecewise planar endpoint trajectories of 3D drawing movements. In two experiments human subjects performed a set of elliptical and figure-8 patterns of different sizes and orientations using their whole arm in 3D. The kinematic characteristics of the endpoint trajectories and the seven joint angles of the arm were analyzed. While the endpoint trajectories produced similar segmentation features as reported in the literature, analyses of the joint angles show no obvious segmentation but rather continuous oscillatory patterns. By approximating the joint angle data of human subjects with sinusoidal trajectories, and by implementing this model on a 7-degree-of-freedom anthropomorphic robot arm, it is shown that such a continuous movement strategy can produce exactly the same features as observed by the above segmentation hypotheses. The origin of this apparent segmentation of endpoint trajectories is traced back to the nonlinear transformations of the forward kinematics of human arms. The presented results demonstrate that principles of discrete movement generation may not be reconciled with those of rhythmic movement as easily as has been previously suggested, while the generalization of nonlinear pattern generators to arm movements can offer an interesting alternative to approach the question of units of action.

1992

Our goal is to understand the principles of Perception, Action and Learning in autonomous systems that successfully interact with complex environments and to use this understanding to design future systems