Towards improved breast cancer detection via multi-modal fusion and dimensionality adjustment
Faseela Chakkalakkal Abdullakutty, Younes Akbari, Somaya Ali Al-Maadeed, Ahmed Bouridane, Iman Mamdouh Talaat, Rifat Hamoudi · Computational and Structural Biotechnology Reports · 2024
A timely and precise diagnosis of breast cancer enhances patient outcomes. Due to its heterogeneity and various pathological manifestations, breast cancer detection requires a multifaceted approach. Deep learning has significantly improved breast cancer diagnosis, especially for image classification. However, it is essential for a comprehensive diagnostic strategy to integrate multimodal data, including histopathology images, clinical reports, and tabular clinical data. The extracted features and their dimensions significantly impact the detection performance in multi-modal fusion, a method commonly used in the literature. Consequently, this article examines the influence of dimensionality on multimodal early fusion approaches for breast cancer detection. For image data, pre-trained features from advanced models VGG-16, ViT, and ResNet-50 are used, while for textual data, BERT is used, along with corresponding Electronic Medical Records (EMR) data. In addition to using denoising auto-encoders, polynomial functions, and neural networks, the dimensionality adjustment method also used Principal Component Analysis (PCA) and auto-encoders to reduce dimensionality. Analysis shows that feature dimensions need to be managed properly to optimize fusion. The results demonstrate that carefully controlling feature dimensions enhances robustness and diagnostic accuracy. Thus dimensionality considerations in multi-modal data integration provide valuable insights for advancing breast cancer detection. • Investigation of dimensionality for multimodal fusion in breast cancer detection. • Evaluation of various dimensionality management methods to optimize fusion. • Provides valuable insights for breast cancer detection on dimensionality in fusion.