Composing complex skills by learning transition policies
Journal
7th International Conference on Learning Representations, ICLR 2019
Date Issued
2019
Author(s)
Abstract
Humans acquire complex skills by exploiting previously learned skills and making transitions between them. To empower machines with this ability, we propose a method that can learn transition policies which effectively connect primitive skills to perform sequential tasks without handcrafted rewards. To efficiently train our transition policies, we introduce proximity predictors which induce rewards gauging proximity to suitable initial states for the next skill. The proposed method is evaluated on a set of complex continuous control tasks in bipedal locomotion and robotic arm manipulation which traditional policy gradient methods struggle at. We demonstrate that transition policies enable us to effectively compose complex skills with existing primitive skills. The proposed induced rewards computed using the proximity predictor further improve training efficiency by providing more dense information than the sparse rewards from the environments. We make our environments, primitive skills, and code public for further research at https://youngwoon.github.io/transition. © 7th International Conference on Learning Representations, ICLR 2019. All Rights Reserved.
Other Subjects
Bipedal locomotion; Continuous control; Initial state; Policy gradient methods; Primitive skills; Sequential task; Training efficiency; Gradient methods
Type
conference paper