Accelerating Real-Time, High-Resolution Depth Upsampling on FPGAs
David Langerman, Sebastian Sabogal, Barath Ramesh, Alan D. George · 2018
While the popularity of high-resolution, computer-vision applications (e.g. mixed reality, autonomous vehicles) is increasing, there have been complementary advances in time-of-flight depth sensor resolution and quality. These advances in time-of-flight sensors provide a platform for new research into real-time, depth-upsampling algorithms targeted at high-resolution video systems with low-latency requirements. This paper describes a case study in which a previously developed bilateral-filter-style upsampling algorithm is profiled, parallelized, and accelerated on an FPGA using high-level synthesis tools from Xilinx. We show that our accelerated algorithm can effectively upsample the resolution and reduce the noise of time-of-flight sensors. We also demonstrate that this algorithm exceeds the real-time requirements of 90 frames per second necessitated by mixed-reality hardware, achieving a lower-bound speedup of 40 times over the fastest CPU-only version.