Abstract A058: Multimodal generative AI jointly learns pathology and clinical data to synthesize a multinational lung cancer cohort
Hanna M. Hieromnimon, V. Miskovic, Matteo Sacco, Alberto Ferrarin, Laura Mazzeo, Andrea Spagnoletti, Monica Ganzinelli, Cecilia Silvestri, Leonardo Provenzano, Claudia Proto, Nir Peled, Enriqueta Felip, Helena Linardou, Martin Reck, Francesco Trovò, Giuseppe Lo Russo, Marina Chiara Garassino, Samantha J. Riesenfeld, Alexander T. Pearson, Arsela Prelaj · Clinical Cancer Research · 2025
Abstract Background: Machine learning models require large, diverse datasets which can be challenging to acquire, even more so for multimodal and paired histology data. Within the I3LUNG European Funded project (NCT05537922), we evaluated multimodal synthetic data generation as a solution to enable domain-specific pretraining and imputation in NSCLC patients treated with immunotherapy (IO) using multimodal data. Methods: Our two-stage method included multimodal data simulation and AI-enabled data evaluation. First, a cross-modal autoencoder jointly embedded histology foundation model features with key clinical features: PD-L1 expression, smoking status, baseline ECOG performance status, histologic subtype, gender, metastatic sites, progression and survival events, LDH, BMI, neutrophil-lymphocyte ratio (NLR), and progression free survival (PFS). The joint latent spaced was sampled using a Gaussian Copula model to generate synthetic patients with coherent multimodal features. Second, to evaluate the fidelity of synthetic clinical representations, we trained a deep neural network models using Cox proportional hazards endpoints on real and simulated data to predict PFS, validating on held-out real patient data. Additionally, we used HistoXGAN to generate paired histology tile images for each synthetic patient. Results: We analyzed NSCLC patients (N=1813) treated with immunotherapy from five centers, split into training (n=1630) and test (n=183) cohorts. The synthetic data matched the original distributions, with minimal differences in continuous features (t-test p > 0.05 and mean differences: BMI -1.72%, PFS -0.37%, LDH -9.18%) and categorical ones (chi-square p > 0.05 and maximum class proportion differences of 3.9%, 15.5%, and 1.2% for bone metastasis, PD-L1 expression, and smoking history respectively). Models trained on synthetic data (N=1000) performed similarly to real data. In validation, the Cox model trained on synthetic data achieved a c-index of 0.683, versus 0.679 for real data (0.6% relative difference). Both synthetic and real data identified consistent prognostic factors (HR [95% CI]): bone metastases (real: 2.36 [1.39-4.80], synthetic: 2.46 [1.39-3.77]), LDH (real: 1.60 [1.16-2.69], synthetic: 1.48 [1.22-2.39]), and liver metastases (real: 1.52 [1.24-3.69], synthetic: 1.46 [1.11-2.80]). Conclusions: Our multimodal synthetic data successfully captured complex multi-feature relationships predictive of PFS in NSCLC patients treated with IO. Synthetic data enables cross-institutional model development while increasing patient privacy, with minimal impact on model performance. This approach paves the way for data democratization, fostering rapid collaboration and mutual validation of AI algorithms. Citation Format: Hanna M. Hieromnimon, Vanja Miskovic, Matteo Sacco, Alberto Ferrarin, Laura Mazzeo, Andrea Spagnoletti, Monica Ganzinelli, Cecilia Silvestri, Leonardo Provenzano, Claudia Proto, Nir Peled, Enriqueta Felip, Helena Linardou, Martin Reck, Francesco Trovo, Giuseppe Lo Russo, Marina Chiara. Garassino, Samantha J. Riesenfeld, Alexander T. Pearson, Arsela Prelaj. Multimodal generative AI jointly learns pathology and clinical data to synthesize a multinational lung cancer cohort [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Artificial Intelligence and Machine Learning; 2025 Jul 10-12; Montreal, QC, Canada. Philadelphia (PA): AACR; Clin Cancer Res 2025;31(13_Suppl):Abstract nr A058.