MA-DyNN: Modal-Adaptive Dynamic Neural Network for Crowd-Counting on Consumer Drones
Hang Shen, Qi Liu, Yu Liu, Tianjing Wang, Guangwei Bai · IEEE Transactions on Consumer Electronics · 2025
Consumer drones are increasingly used for crowd-counting in complex environments; however, their deployment faces challenges from adverse external conditions such as low illumination and inclement weather, as well as inherent limitations like constrained onboard computational resources. To address these constraints, we present MA-DyNN (Modal-Adaptive Dynamic Neural Network), a lightweight and robust framework that dynamically adapts to varying modality conditions for accurate crowd counting. This framework employs an efficient single-stream architecture with specialized modal extractors to capture and integrate complementary information from both visible and thermal infrared (TIR) inputs. Based on the extracted modal features, we design a modality-adaptive gating mechanism to dynamically select the optimal modality based on environmental conditions, favoring visible imagery for inference efficiency in well-lit scenarios and leveraging TIR as auxiliary support under low-light or degraded conditions. To enhance robustness against sensor failure or missing modalities, we develop a density-aware modality converter that adds crowd density constraints to a cycle-consistent generative adversarial learning framework to generate high-fidelity TIR images. This enables consistent performance by aligning synthetic and real TIR-based counting outcomes through adversarial learning. Extensive experiments on DroneRGBT and RGBT datasets show that MA-DyNN achieves superior accuracy, generalization, and real-time performance compared to state-of-the-art multimodal baselines. Its inference acceleration performance approaches single-modality models without compromising the accuracy gains provided by multimodal learning