FPGA-based Accurate Pedestrian Detection with Thermal Camera for Surveillance System
Ryosuke Kuramochi, Masayuki Shimoda, Youki Sada, Shimpei Sato, Hiroki Nakahara · 2019
A surveillance system requires to achieve high accuracy of object detection at all times and to meet real-time processing requirements (30 frames per second) with high energy-efficiency. Since thermal cameras allow to see even in darkness unlike a RGB camera, object detection with a thermal camera obtains higher accuracy in the night, and thereby it has attracted much attention. However, since it is challenging to extract informative features from a thermal image, implementation challenges of an object detection with high accuracy remain. To meet the requirements, we present a sparse YOLOv2-based pedestrian detector with a thermal camera on an FPGA. For high accuracy, we propose a preprocessing that concatenates a thermal image with the background subtracted one for a detector to extract more informative features. Also, we develop a zero weight skipping architecture dedicated to our detector that contains a vectorizing unit that packs successive valid values into the same memory address to realize high parallel degree calculation. It leads to meet the real-time processing requirement with high energy-efficiency. Compared with a conventional one, F-score was 29 points higher, and speed was 3.3 times faster. Therefore, our system is more suitable for surveillance systems.