Graph-based Subtask Representation Learning via Imitation Learning
Se-Wook Yoo, Seung‐Woo Seo · 2022
Many variants of Generative Adversarial Imitation Learning (GAIL) have been successful in the field of imitation learning. However, focusing on a single task cannot solve many mechanical problems, which are composed of multiple sub-tasks with complex relations. Understanding these relations is consistent with identifying the transition order of sub-tasks according to the hierarchical structure. This is because it helps to solve whole problems effectively. Recently, a graph-based neural network (GNN) has been researched popularly for relational inference. In this paper, we focus on solving tasks that require understanding a hierarchical structure of sub-tasks by utilizing the GNN. Our proposed method imitates expert policy and learns sub-task representation simultaneously without predefined task-specific knowledge. Specifically, we effectively combine adversarial imitation learning and sub-task representation learning frameworks. We experiment with our approach on a high-dimensional control task that requires a changeover of sub-tasks to successfully solve whole tasks. Finally, we demonstrate our method has better performance than previous works. Moreover, we visualize the relation of sub-task variables to show how our network understands hierarchical task structure.