DriveRP: RAG and Prompt Engineering Embodied Parallel Driving in Cyber-Physical-Social Spaces
Jun Huang, Hao Ma, Tengchao Zhang, Fei Lin, Siji Ma, Xiao Wang, Fei–Yue Wang · 2024
In recent years, numerous technological advancements in Artificial Generative Intelligences (AGIs) have demonstrated significant potential to transform the intelligence acquisition mechanisms in connected autonomous vehicles (CAVs). Integrating technologies like ChatGPT into CAVs can enhance human-machine interactions. However, the emergence of such new traffic entities may introduce unforeseen hallucinations and complex risks that surpass our current understanding. To address these challenges, Retrieval-Augmented Generation (RAG) and prompt engineering technologies are being explored to enhance the reliability and safety of autonomous driving systems. RAG retrieves relevant contextual information, such as driving experiences and real-time road network status, from external databases to ensure that foundation models have access to accurate and timely data for informed decision-making. Prompt engineering optimizes the performance of large language models in autonomous driving systems by designing and refining prompts that guide the models’ responses, thereby improving their relevance and accuracy in various driving scenarios. Together, these technologies enhance the robustness and trustworthiness of autonomous driving systems. This paper proposes DriveRP, a framework that integrates RAG and prompt engineering within the Descriptive-Predictive-Prescriptive Intelligence framework of Parallel Driving theory. DriveRP aims to enhance the safety and interpretability of autonomous vehicle trajectory planning, decision-making, and motion control, ultimately achieving the "6S" goals. Grounded in Digital Twins and Metaverse-embodied parallel driving theory, DriveRP provides the infrastructure and foundational intelligence for parallel driving with Multi-modal Large Lange Models(MLLMs). Additionally, the paper discusses future trends and potential research directions, focusing on the "6S" goals of parallel driving: Smart, Safe, Secure, Sensitive, Sustainable, and Serviceable.