Decoupling and Aligning Modality-Shared Semantics for Single-Domain Generalization in Hyperspectral Image Classification
Xi Chen, Maojun Zhang, Shen Yan, Yuxiang Liu, Chen Chen · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · 2025
Leveraging natural language to guide visual models for single-domain generalization (SDG) in hyperspectral image classification (HSIC) is a promising yet challenging frontier. A fundamental obstacle is the inherent heterogeneity between visual and linguistic modalities, where modality-specific attributes interfere with direct feature alignment, consequently limiting the generalization capacity of the hyperspectral encoder. To overcome this, we propose a novel framework, Decoupling and Aligning Modality-Shared Semantics (DAMS). The core innovation of DAMS is a feature disentanglement strategy that isolates modality-shared semantics from modality-specific attributes for both spectral and textual data. This allows for a more precise and robust alignment of the shared components in a common embedding space via modality-shared supervised contrastive learning. This core alignment is further regularized by two auxiliary objectives: a bidirectional cross-modal transformation task to enforce semantic consistency, and an intra-modal contrastive loss to ensure feature invariance across data augmentations. Extensive experiments on three challenging cross-domain HSIC benchmarks validate our approach, demonstrating that DAMS establishes a new state-of-the-art by significantly outperforming existing methods. Ablation studies further confirm that our disentangle-and-align paradigm is the primary driver of the model's superior generalization performance. The code is available athttps://github.com/daxichen/DAMS.