Robust Weather Recognition in Images via Interactive CNN-Transformer
Tianyu Liu, Qingwen Hou, Jie Zhang, Xianzhong Chen, Jinghui Cheng · 2023
Weather recognition is a fundamental yet challenging task for the multi -sensor perception system of driverless vehicles. It is difficult for current joint perception framework to autonomously recognize weather scenes and adaptively adjust the weight of camera involved in perception. This paper proposes a weather recognition method with a new interactive CNN - Transformer architecture to learn global and local contextual information from images. The framework starts by deepening and widening the convolution blocks to obtain multi-scale local scene features. We re-weight different feature channels by channel-attention mechanism. Then, a new interactive feature fusion network between convolutional structure and stacked Transformer is developed, which can gradually complement the global feature information lost in CNN. We evaluated the proposed method on a public weather image dataset. The recognition accuracy reaches 95.14%, which improves 5.68% and 6.15% compared with the ResNet-50 and ViT16 and achieves state-of-the-art performance on the experimental dataset. At the same time, the comprehensive evaluation of recall and precision show that the proposed method is more robust when dealing with multi-class weather recognition tasks.