Cloud Classification Method Based on Convolutional Encoder and Multimodal Feature Fusion
Yanfeng Li, Man Xu, Ming Fang · 2022
To overcome the challenges posed by the lack of uniqueness, limited categories, and inaccurate representations of clouds, this paper has presented a cloud classification method based on convolutional encoder and multimodal feature fusion. Firstly, to extract the most representative cloud features, we designed a Cloud-SSAE model, which adds a convolution neural network on top of the autoencoder learning mechanism, incorporates a penalty term within the hidden layer to promote sparsity in the subsequent input layer, and utilizes max-pooling to increase the robustness of the features. Secondly, we improved VGG16 model by incorporating dilated convolution to obtain dense local features. We further fused the color, texture, morphology, temperature, and humidity into the multimodal features. Lastly, we combined Cloud-SSAE model features, VGG16 model features, and multimodal features, and input the fused features into Support vector machine for classification. Consequently, the method proposed herein has the capacity to facilitate the classification of 11 distinct cloud types, encompassing but not restricted to cirrus and cumulonimbus cloud, with an improved accuracy level of 92.1%.