Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3D Object Detection
Shihao Wang, Xiaohui Jiang, Ying Li · IEEE Transactions on Intelligent Vehicles · 2023
The dominant multi-camera 3D detection paradigm is based on explicit 3D feature construction, which requires complicated indexing of local image-view features via 3D-to-2D projection. Other methods implicitly introduce geometric positional encoding and perform global attention (e.g., PETR) to build the relationship between image tokens and 3D objects. The 3D-to-2D perspective inconsistency and global attention lead to a weak correlation between foreground tokens and queries, resulting in slow convergence. We proposeFocal-PETRwith instance-guided supervision and spatial alignment module to adaptively focus object queries on discriminative foreground regions.Focal-PETRadditionally introduces a down-sampling strategy to reduce the consumption of global attention. Our model achieves leading performance on the large-scale nuScenes benchmark and a superior speed of30 FPSon a single RTX3090 GPU. Extensive experiments show that our method outperforms PETR while consuming3xfewer training hours. The code is made publicly available.