1282 H&E 2.0 virtual staining of CD3+ T cells: improving cell labeling for precise model training
Xinyun Feng, Marcia Zhang, Willa Wen‐You Yim, Felicia Yu Ting Wee, Menaka Priyadharsani Rajapakse, Jeffrey Chun Tatt Lim, Chan Way Ng, Inti Zlobec, Bernett Teck Kwong Lee, Olaf Rotzchke, Joe Poh-Sheng Yeong, Mai Chan Lau · Regular and Young Investigator Award Abstracts · 2023
Background Advancement in multiplex immunohistochemistry (mIHC) allows for single-cell tumor microenvironment analysis, greatly enhancing our comprehension of cancer immunology.1 However, clinical application of mIHC remains challenging due to technical variations and cost. Hematoxylin and eosin (H&E) staining, a cost-effective method commonly used in clinics, combined with emerging digital pathology,2 offers promising prospects. Increasing evidence suggests that deep learning (DL) can discern molecular variations from histomorphological features in H&E images.3 4 This has sparked research into applying DL to digital H&E images, a concept referred to as H&E 2.0.5 Model training demands accurate cell labels, but traditional manual annotation methods are labor-intensive, time-consuming, and show significant observer variability.3 6 In this study, we investigate and propose the best methods for image registration and nuclear segmentation to create accurate cell labels for training a predictive model particularly for CD3+ T-cell prediction. Our primary aim is to create an automated workflow that enables robust and precise virtual staining on widely used H&E images. Methods The cohort includes 19 triple-negative breast cancer samples in a TMA format. The TMA slide underwent mIHC staining, was subsequently washed, stained with H&E, and imaged. The mIHC staining panel consists of DAPI/CD20/CD3/Ki67/CD163/FOXP3 stains. For each of the samples, the H&E and mIHC stained images were first cropped into individual core areas. Using SIFT from OpenCV (v4.7.0.72), the mIHC images were warped to align with their corresponding H&E images. Nuclear segmentation on H&E images was performed using the pretrained ‘2D_versatile_he’ model from StarDist (v0.8.3) where various probability thresholds were assessed (figure 1). All analysis was performed using Python programming. Results Our findings indicate that the use of SIFT registration significantly enhances the precision of mIHC signal mapping to H&E images, both at the tissue and cellular levels. This is observed through the alignment of the nuclear mask derived from H&E images with the DAPI nuclear signal in mIHC (figure 2). Using the StarDist model on H&E images at a 0.7 probability threshold gave results closest to manual annotation, unlike the over-segmentation seen with DeepCell’s Mesmer algorithm (figure 3). Conclusions We have presented an optimized approach for generating ground truth with minimal user involvement. This approach can be effortlessly scaled to multiple biomarkers and an increased quantity of training images. In future work, we plan to broaden our focus to include other types of tumors and evaluate the impact on the performance of virtual staining predictions. Acknowledgements This work was supported by the Bioinformatics Institute, Singapore Immunology Network, and Insitute of Molecular and Cell Biology, Agency for Science, Technology and Research (A*STAR). References Sahardi VK, Armengol G. Molecular Biomarkers in Cancer. Biomolecules. 2022;12(8):1021. Lee K, Lockhart JH, Xie M, Chaudhary R, Slebos RJC, Flores ER, Chung CH, Tan AC. Deep Learning of Histopathology Images at the Single Cell Level. Front Artif Intell. 2021;4:754641. Liu Y, Li X, Zheng A, Zhu X, Liu S, Hu M, Luo Q, Liao H, Liu M, He Y, Chen Y. Predict Ki-67 Positive Cells in H&E-Stained Images Using Deep Learning Independently From IHC-Stained Images. Front Mol Biosci. 2020;7:183. Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyo D, Moreira AL, Razavian N, Tsirigos A. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med. 2018;24(10):1559–1567. Comiter C, Vaishnav ED, Ciampricotti M, Li B, Yang Y, Rodig SJ, Turner M, Pfaff KL, Jané-Valbuena J, Slyper M, Waldman J, Vigneau S, Wu J, Blosser TR, Segerstolpe Å, Abravanel D, Wagle N, Zhuang X, Rudin CM, Klughammer J, Rozenblatt-Rosen O, Kobayash-Kirschvink KJ, Shu J, Regev A. Inference of single cell profiles from histology stains with the Single-Cell omics from Histology Analysis Framework (SCHAF). bioRxiv. 2023:2023.03.21.533680. Couture HD. Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review. J Pers Med. 2022;12(12):2022.