A Tale of Two DRAGGNs: A Hybrid Approach for Interpreting Action-Oriented and Goal-Oriented Instructions
Siddharth Karamcheti, Edward Williams, Dilip Arumugam, Mina Rhee, Nakul Gopalan, Lawson L. S. Wong, Stefanie Tellex · 2017
Robots operating alongside humans in diverse, stochastic environments must be able to accurately interpret natural language commands.These instructions often fall into one of two categories: those that specify a goal condition or target state, and those that specify explicit actions, or how to perform a given task.Recent approaches have used reward functions as a semantic representation of goal-based commands, which allows for the use of a state-of-the-art planner to find a policy for the given task.However, these reward functions cannot be directly used to represent action-oriented commands.We introduce a new hybrid approach, the Deep Recurrent Action-Goal Grounding Network (DRAGGN), for task grounding and execution that handles natural language from either category as input, and generalizes to unseen environments.Our robotsimulation results demonstrate that a system successfully interpreting both goaloriented and action-oriented task specifications brings us closer to robust natural language understanding for human-robot interaction.