Virtual multiplex immunofluorescence identifies lymphocyte subsets predictive of response to neoadjuvant therapy
Anran Li, Madeleine S. Torcasso, Anna H. Woodard, Hanna M. Hieromnimon, Jonathan Trujillo, Long C. Nguyen, Margarite D. Matossian, James M. Dolezal, Rebecca Abraham, Marcus R. Clark, Galina F. Khramtsova, Yuanyuan Zha, Maryellen Lissak Giger, Alexander T. Pearson, Frederick Matthew Howard · Therapeutic Advances in Medical Oncology · 2025
Background: Hematoxylin and eosin (H&E) staining is routine in pathology but lacks cellular specificity. Multiplex immunofluorescence (mIF) captures spatial immune relationships in tumors, but cost and complexity limit clinical application. Novel approaches to yield similar information from readily available tumor histology are needed. Objectives: Develop and validate a novel deep learning tool capable of translating standard H&E-stained histopathology images into high-fidelity synthetic mIF images that preserve immune cell information predictive of treatment response in breast cancer. Design: Comparative model evaluation and predictive modeling in a retrospective breast cancer cohort. Methods: Core-needle biopsies from 17 triple-negative breast cancer cases underwent mIF imaging. Hematoxylin and eosin and mIF images for DAPI (nuclei), pan-CK (tumor), CD3/CD4/CD8 (T-cells), and CD20 (B cells) were aligned. A pipeline outperforming standard Pix2Pix and CycleGAN image translation networks was developed, "multiplex Synthetic Immunofluoresence Generated through H&E Translation" (mSIGHT), which integrates a registration network to overcome misalignment between the input and target images. Generated images were evaluated with pixel-level metrics and biological metrics, including cell density and cell-to-cell adjacency. The pipeline was then applied to an external cohort to assess associations between predicted immune features and pathologic response to neoadjuvant chemotherapy. Results: = 0.002), independent of receptor status, grade, and pathologist TIL annotations. Conclusion: The mSIGHT pipeline enables translation of routine H&E slides into virtual mIF images with interpretable immune biomarkers, offering a scalable and affordable alternative to multiplex imaging. It also identifies immune features predictive of therapeutic response and has the potential to assist in the personalization of neoadjuvant therapy.