Emergent proximo-distal maturation through adaptive exploration
Freek Stulp, Pierre‐Yves Oudeyer · 2012
Life-long robot learning in the high-dimensional real world requires guided and structured exploration mechanisms. In this developmental context, we investigate here the use of the recently proposed PICMAES2episodic reinforcement learning algorithm, which is able to learn high-dimensional motor tasks through adaptive control of exploration. By studying PICMAES2in a reaching task on a simulated arm, we observe two developmental properties. First, we show how PICMAES2autonomously and continuously tunes the global exploration/exploitation tradeoff, allowing it to re-adapt to changing tasks. Second, we show how PICMAES2spontaneously self-organizes a maturational structure whilst exploring the degrees-of-freedom (DOFs) of the motor space. In particular, it automatically demonstrates the so-called proximo-distal maturation observed in humans: after first freezing distal DOFs while exploring predominantly the most proximal DOF, it progressively frees exploration in DOFs along the proximo-distal body axis. These emergent properties suggest the use of PICMAES2as a general tool for studying reinforcement learning of skills in life-long developmental learning contexts.