Aerial Photograph Categorization by Cross-Resolution Deep Human Gaze Behavior Learning
Luming Zhang, Ming Chen, Guifeng Wang, Zhiming Wang · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · 2022
Accurately recognizing aerial photographs is a useful technique in many domains like autonomous driving and environmental evaluation. In practice, both low-resolution and high-resolution aerial photos are captured asynchronistically for each region, as there are hundreds of earth observation satellites orbitting the earth. Realizing such multi-resolution-based region recognition is a difficult task due to three challenges: 1) mimicking human visual perception when they actively viewing the semantic objects inside each aerial photo; 2) deeply modeling the visually/semantically salient objects sequentially perceived by human visual system; 3) developing a cross-resolution knowledge transferal module to enhance the feature representation for an area. To solve these challenges, we propose a cross-domain aerial photograph recognition system by leveraging the low-resolution spatial composition to enhance the deep encoding of human gaze shifting path (GSP) with a high-resolution. More specifically, we first use an active learning algorithm to discover multiple visually/semantically salient object patches for constructing GSP from a high-resolution aerial photo. Then, an aggregation-based deep model is formulated to sequentially link the deep features learned from the object patches inside each GSP. Subsequently, a novel knowledge transferal algorithm leverages the global spatial composition from low-resolution counterparts to upgrade the deeply-learned GSP feature of the high-resolution aerial photo. Using the upgraded deep GSP feature, a multi-label SVM classifier is trained for categorizing aerial photographs. Comparative studies on our million-scale aerial photograph set have demonstrated the competitiveness of our approach.