IR-Setnet: Sparsity Aware Ensemble Network for Infrared Imaging Based Drone Localization and Tracking In Distorted Surveillance Videos
Dharini Raghavan, S. Sethu Selvi, S Raghuram, Vishnu R Rao, A Srinidhi · 2023
Although Unmanned Aerial Vehicles (UAVs/drones) have been extensively deployed for various applications, unauthorized operations cause serious security threats and infringement of privacy. Target (drone) localization plays a crucial role in military operations that is extremely challenging under low visibility conditions, uneven illumination, weak contrast, background distortions and close resemblances to other flying entities such as birds. In such cases, infrared imaging outperforms RGB imaging since the images in the former case are based on temperature differences and they help identify and track objects that are not visible in RGB images. In this paper, a sparse ensemble network is proposed for infrared drone localization combined with a CLIP-based zero shot tracking network for tracing the trajectory of the target. Static pruning and model quantization are adopted for introducing sparsity in the ensemble network. This is further combined with a Vision Transformer (ViT) for multi-class distortion classification in real-time surveillance videos. The end-to-end pipeline is termed as IR-SETNET which exhibits 25% improvement in target localization with a mean average precision (mAP) score of 99.90%, drone tracking accuracy of 95.60%, mean distortion classification accuracy of 89.75% and a five-fold improvement in inference speed on introduction of sparsity.