SimLane: A Risk-Orientated Benchmark for Lane Detection
Xinyang Zhang, Zhisheng Hu, Shengjian Guo, Zhenyu Zhong, Kang Li · 2022
Lane detection, which detects lanes on the road from a camera frame, plays an essential role in the successful and safe application of autonomous driving. A few previous works explore their robustness to adversarial road conditions (e.g., dirty patch [1]); however, to the best of our knowledge, no prior work systematically studies their robustness to common environmental variations, like weather, lighting, etc. Those variations will inevitably happen to autonomous vehicles worldwide, owning to the long-tail nature of driving. Therefore, it is critical to understand whether state-of-the-art lane detection models are sensitive to such environmental variations. This paper fills this research gap by collecting a dataset of variations from simulations and benchmarking state-of-the-art models against this new dataset.We first systematically investigate and categorize environmental variations that might incur negative impacts against lane detection based on existing datasets and online driving clips. We then develop a pipeline to collect driving clips under these variations from simulations. The dataset we collected, called SimLane, consists of 149 driving clips of eight different variations that could also serve as a baseline for future research on lane detection safety. Our empirical results verify that state-of-art models suffer from sensitivity to environmental variations from a minor scale to a vast scale. In particular, road graffiti with a medium dense level will cause the detection accuracy of SCNN model ([2], trained on TuSimple Dataset [3]) to drop from 0.897 to 0.219.