NSCLC whole slide image survival analysis by end to end transformer
Guokai Fu, Peng Xia · 2023
Whole Slide Images (WSI) are commonly used in clinical diagnosis. After digitizing pathological slides, they become enormous gigapixel images, typically lacking pixel-level annotations. This poses significant challenges for employing end-to-end deep learning models for learning and predictions, especially in tasks like survival analysis that are inherently more challenging due to censored data.In this study, we utilized a Resnet50 model pretrained on ImageNet to extract features from WSIs, facilitating data dimensionality reduction. We proposed a training approach for an end-to-end model where, after calculating the gradient for an individual WSI, gradients from multiple images were accumulated for backpropagation. This enabled the successful training of an end-to-end transformer model. Results from training and testing on 203 WSIs with corresponding labels demonstrate the reliability of this training method. Unlike typical classification problems, the presence of missing data in survival analysis makes our proposed approach particularly robust.