Learning the skill of archery by a humanoid robot iCub

Humanoid robot iCub learns the skill of archery. After being instructed how to hold the bow and release the arrow, the robot learns by itself to aim and shoot arrows at the target. It learns to hit
…

Humanoid robot iCub learns the skill of archery. After being instructed how to hold the bow and release the arrow, the robot learns by itself to aim and shoot arrows at the target. It learns to hit the center of the target in only 8 trials.

The problem of detecting where the target is, and what isthe relative position of the arrow with respect to the centerof the target, is solved by image processing. We use colorbaseddetection of the target and the tip of the arrow basedon Gaussian Mixture Model (GMM). The color detection isdone in YUV color space, where Y is the luminance, andUV is the chrominance. Only U and V components are usedto ensure robustness to changes in luminosity.In a calibration phase, prior to conducting an archeryexperiment, the user explicitly defines on a camera imagethe position and size of the target and the position of thearrow’s tip. Then, the user manually selects NT pixels lyinginside the target in the image, and NA pixels from the arrow’stip in the image. The selected points produce two datasets:cT 2 R2NT and cA 2 R2NA respectively.From the two datasets cT and cA, a Gaussian MixtureModel (GMM) is used to learn a compact model of the colorcharacteristics in UV space of the relevant objects. EachGMM is described by the set of parameters fk; k;kgKk=1,representing respectively the prior probabilities, centers andcovariance matrices of the model (full covariances are consideredhere). The prior probabilities k satisfy k 2 R[0;1]andPKk=1 k = 1. A Bayesian Information Criterion (BIC)[13] is used to select the appropriate number of GaussiansKT and KA to represent effectively the features to track.After each reproduction attempt, a camera snapshot istaken to re-estimate the position of the arrow and the target.2From the image cI 2 R2NxNy of NxNy pixels in UVcolor space, the center m of each object on the image isestimated through the weighted sum

2.
MotivationHow a robot can learn complex motor skills?Why archery task?bi-manual coordinationintegration of image processing, motor control and learning parts in one coherent taskusing tools (bow and arrow) to affect an external object (target)appropriate task for testing different learning algorithms, because the reward is inherently defined by the goal of the taskPetar Kormushev, Italian Institute of Technology (IIT)2/20

10.
PoWER - implementationPolicy parameters :relative position the two hands(3D vector from right to left hand)Policy update rule:Importance samplinguses best σ rollouts so farrelative explorationPetar Kormushev, Italian Institute of Technology (IIT)8/20