Object-focused Risk Evaluation of AI-driven Perception Systems in Autonomous Vehicles
Subhadip Ghosh, Aydin Zaboli, Junho Hong, Jaerock Kwon · 2024
One of the primary motivations for autonomous vehicle (AV) technology is to reduce road accidents compared to human-driven cars. This necessitates having robust perception systems to detect and classify objects correctly in real-time environments. Various factors, including the complexity of the scene, the type of object, the capability of the perception sensors, and the performance of AI-based algorithms, can affect its robustness. Furthermore, vulnerabilities in these factors can be exploited as cyber-physical attacks. Hence, this paper presents a novel mathematical model for system-level risk evaluation of AV perception systems that incorporates the relevant objects for AV applications and the machine learning (ML) algorithms used to detect and classify them. This model is adapted from the ISO/SAE 21434 threat analysis and risk assessment (TARA) model with an enhancement in impact rating and attack feasibility assessment. Additionally, a case study for impact rating is demonstrated with real data from traffic crashes where the most important objects are impacted. Also, the effect of the robustness of the detection algorithm on attack feasibility assessment is illustrated with some AI/ML-based state-of-the-art detection algorithms used in AVs.