Fine-Tuned Large Language Model for Autonomous Vehicles Accident Report
Feng Yao, Xingyu Huang, Zhixiang Zhang · 2024
In recent years, with the rapid development of large language models, the application of artificial intelligence in various fields has made remarkable progress, especially in the research and development and application of autonomous driving technology, this trend is particularly obvious. In this study, Low-Rank Adaptation (LoRA) technology and BitsAndBytes quantization technology were used to fine-tune LLaMA-2 7B model efficiently. The introduction of these techniques is designed to reduce the dependence on computing and memory resources during model fine-tuning, making it possible to effectively optimize large pretrained models even in resource-limited environments. The experimental results show that our fine-tuned model achieves the best effect in the text generation task, with the overall cosine similarity reaching 0.740 and Jaccard similarity reaching 0.620, which can accurately perform the text generation task, proving the effectiveness of the fine-tuned model. It also provides a new idea for applying large language model to autonomous driving accident analysis in the future.