The ABCs of Assured Autonomy
Joshua M. Mueller · 2019
Each passing day seems to bring new instances of automation of routine tasks and the addition of artificial intelligence or machine learning algorithms to new domains. Autonomous systems are a diverse class of technologies ranging from AI driven natural language processing and image recognition to closed-form control systems for aircraft autopiloting. Coincident with this explosion in autonomy is a related drumbeat of stories of AI failures. These failures have ranged from tragedies resulting in bystander deaths to humorous examples of video game flaws being exploited. Widespread deployment and use of autonomous systems will depend on society trusting that these systems will perform as expected. While bias from incomplete training data is a well-trod area for improvement toward reducing AI failures, it is unclear if this bias is a sufficient or merely a necessary condition for loss of trust. What else may be needed for public assurance in autonomous systems? Here we identify three features of diverse autonomous systems that serve as a foundation for assured autonomy. These features are: the accuracy with which the algorithm senses and perceives the environment in a manner relatable to humans; a reduction in bias driven by the training data and algorithmic bias; and the complexity of the algorithmic process in terms of the ability to reverse engineer the decision-making processes. Building from this foundation, future autonomous systems can begin to reverse the loss of trust starting to be seen with respect to these technologies.