A ConvNeXt-Transformer Approach for Automated Conclusion Generation from Mammography
Eduardo de Ávila-Armenta, Beatriz A Bosques-Palomo, José G. Tamez‐Peña, Mario A Monsivais-Molina, Mario A Monsivais-Molina, Jorge Alberto Garza-Abdala, Sadam Hussain, Servando Cardona‐Huerta, Daly B Avenda Ño-Avalos, Daniel Vela-Jarquin · 2025
Breast cancer remains a critical global health concern, particularly in low-resource settings where limited access to radiologists delays diagnosis. While digital mammography enables early detection, its effectiveness is often constrained by radiological expertise shortages. This study proposes a multimodal deep learning system to automate the generation of radiology report conclusions from mammogram images, supporting diagnostic decision-making and reducing radiologist workload. An encoder-decoder architecture is implemented, combining a ConvNeXt model for image feature extraction with a Transformer-based decoder for text generation. The model is trained and evaluated on the RSNA Screening Mammography dataset and a curated dataset from Tec Salud in Mexico. Focusing on BI-RADS classification and biopsy recommendations, the system achieves high-quality text generation (BLEU, ROUGE-L, METEOR up to 0.98) and substantial agreement with expert assessments (Cohen's kappa of 0.93). Performance remains fair to moderate for more complex categories. The system shows strong potential as a prioritization tool in screening campaigns, particularly in underserved regions, and offers a scalable solution adaptable to other diagnostic imaging tasks.