Survival Prediction for Gastric Cancer via Multimodal Learning of Whole Slide Images and Gene Expression
Yuzhang Xie, Guoshuai Niu, Qian Da, Wentao Dai, Yang Yang · 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · 2022
Gastric cancer (GC) is one of the most common malignancies worldwide. As histopathology tissue analysis is considered as the gold standard in cancer studies, whole slide images (WSIs) have been widely used for GC diagnosis and prognosis, while multimodal studies for GC patients have been very few. Especially, WSIs and gene expression are complementary modalities of data, thus fusion of these two modalities has great potential in the prediction of survival outcomes and other computer-aided tasks, like the mechanism study and clinical treatment for GC patients. However, multimodal learning requires good data fusion strategies and also suffers from the missing data issue. To address these issues, we propose GC-SPLeM, to predict risk scores for patients, which consists of three parts, WSI feature extraction, modal-fusing network, and GNN-based predictor. We conduct experiments on a GC dataset built by ourselves and a public dataset for survival prediction. For both datasets, GC-SPLeM outperforms the state-of-the-art single-modality learning method and multimodal learning method by large margins (over 5% on C-index). We find that the GNN plays an important role in performance enhancement. Through learning the graph of patients, topological structure and neighborhood clinical information are encoded into feature representations of patients. GC-SPLeM not only improves the survival prediction results but also has advantages in dealing with incomplete data over other methods. The Source code, sample data, and gene list of this study are available at https://github.com/constantjxyz/GC-SPLeM.