Online Behavior Modification for Expressive User Control of RL-Trained Robots
Isaac S. Sheidlower, Mavis Murdock, E. Wes Bethel, Reuben M. Aronson, Elaine Schaertl Short · 2024
Reinforcement Learning (RL) is an effective method for robots to learn tasks. However, in typical RL, end-users have little to no control over how the robot does the task after the robot has been deployed. To address this, we introduce the idea of online behavior modification, a paradigm in which users have control over behavior features of a robot in real-time as it autonomously completes a task using an RL-trained policy. To show the value of this user-centered formulation for human-robot interaction, we present a behavior-diversity--based algorithm, Adjustable Control Of RL Dynamics (ACORD), and demonstrate its applicability to online behavior modification in simulation and a user study. In the study (n =23), users adjust the style of paintings as a robot traces a shape autonomously. We compare \algoshort to RL and Shared Autonomy (SA), and show \algoshort affords user-preferred levels of control and expression, comparable to SA, but with the potential for autonomous execution and robustness of RL. The code for this paper is available at https://github.com/AABL-Lab/HRI2024_ACORD