Towards Trust: An Analysis of a Deep Learning Tracker Agent
Devon Hood, Micah Bryant, Anastacia MacAllister, Rey Nicolas · 2024
Rapid progress in the fields of Artificial Intelligence and Robotics is resulting in increasingly complex collaborative interactions between human and machines. A result of these emerging collaborations, human machine teaming (HMT) is becoming a topic of great interest. With the notion of a man-machine team, the desire to produce understandable autonomy is becoming an important topic for researchers to ensure this technology is deployed safely and intelligently. Unfortunately, the pace of HMT research has not advanced in line with development of autonomous systems. In this paper we begin to address these challenges by quantifying the performance of a reinforcement learning based Track-Avoid agent, under varying simulated stress tests. Our experimental design includes an analysis of our agent’s robustness against adversaries with increasing difficulty and tactical skill. Ultimately, our series of stress tests aim to help the human operator understand: Did we win? If not, why? This understanding can then help them identify where greater human supervisory control may be needed within the HMT, resulting in safer, more effective system employment. Our initial stress test findings show our reinforcement learning Track-Avoid agent maintains higher overall robustness by a metric of skill degradation when compared to our baseline heuristic actor, and thus, achieves higher win rates. However, we also find that for some adversary tactics our agent spends far too much time maneuvering and not tracking, indicating a human operator may need to help their autonomous counterpart instigate tracking by navigating to an advantageous starting position. In the end, our methods and results help describe to the community a quantitative strategy for helping to increase operator understanding of when and how to successfully employ human machine teaming.