Two-BranchTGNet: A Two-Branch Neural Network for Breast Cancer Subtype Classification
Jiahui Yu, Hongyu Wang, Yingguang Hao · 2023
A large amount of information can be obtained from pathology images, and in order to make full use of the information on the images, we designed a two-branch neural network. One branch uses Vision Transformer to obtain pixel-level information; the other branch introduces knowledge of graph theory to construct a spatial graph structure of the tumor microenvironment and uses a graph neural network to capture spatial geometric information. For this purpose, we designed a weighted tensor feature fusion module to fuse the information of the two modalities. A smoothed labelling loss function was used to improve the accuracy of breast cancer subtype classification. Compared to the single-network backbone, the model achieved an accuracy improvement of over 6% on the BRACS dataset and a 1% improvement on the BACH dataset. The dual-branch approach outperforms the single-network backbone method and effectively integrates image-level morphological features and spatial features.