Real-Time Graph-based Point Cloud Networks on FPGAs via Stall-Free Deep Pipelining
Marc Neu, I. Haide, Timo Justinger, Till Rädler, Valdrin Dajaku, T. Ferber, Jürgen Becker · 2025
Graph-based Point Cloud Networks (PCNs) are powerful tools for processing sparse sensor data with irregular geometries, as found in high-energy physics detectors. However, deploying models in such environments remains challenging due to stringent real-time requirements for both latency, and throughput. In this work, we present a deeply pipelined dataflow architecture for executing graph-based PCNs on FPGAs. Our method supports efficient processing of dynamic, sparse point clouds while meeting hard real-time constraints. We introduce specialized processing elements for core graph operations, such as GraVNet convolution and condensation point clustering, and demonstrate our design on the AMD Versal VCK190. Compared to a GPU baseline, our FPGA implementation achieves up to $5.25 \times$ speedup in throughput while maintaining latencies below $10 \mu \mathrm{~s}$, satisfying the demands of real-time trigger systems in particle physics experiments. An open-source reference implementation is provided.