Towards Modular Multispectral Object Detection using Adaptive Weighted Fusion
Rohan Nitin Pradhan · TSpace (University of Toronto) · 2020
Object detection is a common computer vision task, used for perception for robotic applications and autonomous vehicles. While many deep learning-based approaches have improved the accuracy and performance of these systems, they lack the environmental robustness to be reliably deployed in critical applications. We propose a novel multispectral, modular object detection architecture to improve the environmental robustness of these systems using an adaptive approach. Our approach is modeled after the multimodal nature of human perception, and it's ability to dynamically weigh senses given the environment. Furthermore, our architecture is modular/model agnostic allowing for simpler deployment. We evaluate our architecture on two datasets with a wide-range of simulated environmental perturbations. Our architecture manages to improve robustness in these conditions and reduces variance in performance regardless of the environment, reducing the sample variance of the mAP between experimental scenarios to ~0.0012, compared to ~0.111 on the early fusion architecture.