A 38.5TOPS/W Point Cloud Neural Network Processor with Virtual Pillar and Quadtree-based Workload Management for Real-Time Outdoor BEV Detection
Sukbin Lim, Jaehoon Heo, Jinho Yang, Joo-Young Kim · 2024
In recent years, the significance of 3D point cloud processing has witnessed a remarkable upsurge, particularly in emerging application domains such as autonomous vehicles and mixed reality. The voxel-based methodologies, paired with point cloud neural network (PNN), are dominant for managing large-scale outdoor point cloud. However, the adoption of voxel introduces significant computational and memory overhead due to their cubic structure. To address this challenge, recent studies have introduced the pillar, which extends a voxel vertically to offer a bird's-eye-view (BEV) representation while reducing computational requirements significantly. As shown in Fig. 1, pillar-based PNN [1]–[2] comprises two key stages: the feature encoding network (FEN) and the convolutional neural network (CNN) backbone. While previous accelerators [3–4] have primarily focused on accelerating the backbone, the significance of FEN is emphasized as its processing time becomes on par with that of the backbone. There are major computational challenges that previous works have not fully addressed. 1) FEN necessitates a large on-chip memory due to the unpredictable access pattern of point cloud and dependency of cluster for handling dynamic real-world environments. 2) The pseudo-image derived from FEN exhibits high sparsity due to the limitations of LiDAR, which cannot capture the back side of objects once they are detected. However, its sparsity diminishes across layers during operations.