Neural program synthesis from diverse demonstration videos
Journal
35th International Conference on Machine Learning, ICML 2018
Journal Volume
11
Pages
7614-7623
Date Issued
2018
Author(s)
Abstract
Interpreting decision making logic in demonstration videos is key to collaborating with and mimicking humans. To empower machines with this ability, we propose a neural program synthesizer that is able to explicitly synthesize underlying programs from behaviorally diverse and visually complicated demonstration videos. We introduce a summarizer module as part of our model to improve the network's ability to integrate multiple demonstrations varying in behavior. We also employ a multi-task objective to encourage the model to learn meaningful intermediate representations for end-to-end training. We show that our model is able to reliably synthesize underlying programs as well as capture diverse behaviors exhibited in demonstrations. © 2018 by the Authors All rights reserved.
Other Subjects
Artificial intelligence; Decision making; Learning systems; End to end; Intermediate representations; Program synthesis; Demonstrations
Type
conference paper