Recognizing Epithelial Cells in Prostatic Glands Using Deep Learning
Liton Devnath, Puneet Arora, Anita Carraro, Jagoda Korbelik, Mira Keyes, Gang Wang, Martial D. Guillaud, Calum E. MacAulay · Cells · 2025
Artificial intelligence (AI) is becoming an integral part of pathological assessment and diagnostic procedures in modern pathology. As most prostate cancers (PCa) arise from glandular epithelial tissue, an AI-based methodology has been developed to recognize glandular epithelial nuclei in prostate biopsy tissue. An integrated machine-learning network, named GlandNet, was developed to correctly recognize the epithelial cells within prostate glands using cell-centric patches selected from the core biopsy specimens. Feulgen-Thionin (a DNA stoichiometric label) was used to stain biopsy sections (4–7 µm in thickness) from 82 active surveillance patients diagnosed with PCa. Images of these sections were human-annotated, and the resultant dataset consisted of 1,264,772 segmented, cell-centric nuclei patches, of which 449,879 were centered on epithelial gland nuclei from 110 needle biopsies (training set: n = 66; validation set: n = 22; and test set: n = 22). The training of GlandNet used semi-supervised machine-learning knowledge of the training and validation cohorts and integrated both human and AI predictions to enhance its performance on the test cohort. The performance was evaluated against a consensus deliberation from three observers. The GlandNet demonstrated an average accuracy, sensitivity, specificity, and F1-score of 94.1%, 95.7%, 87.8%, and 95.2%, respectively, when tested on the 20,735 glandular cells found in the three needle biopsies with the visually best consensus predictions. Conversely, the average accuracy, sensitivity, specificity, and F1-score were 90.9%, 86.4%, 94.0%, and 89.7% when assessed on 57,217 cells found in the three needle biopsies with the visually worst consensus predictions. GlandNet is a first-generation AI with an excellent ability to differentiate between epithelial and stromal nuclei in core biopsies from patients with early prostate cancer.