Optimizing Medical Image Analysis: A Performance Evaluation of YOLO-Based Segmentation Models
Haifa A. Alanazi · International Journal of Advanced Computer Science and Applications · 2025
Instance segmentation is a critical component of medical image analysis, enabling tasks such as tissue and organ delineation, and disease detection. This paper provides a detailed comparative analysis of two fine-tuned one-stage object detection models, YOLOv11-seg and YOLOv9-seg, tailored for instance segmentation in medical imaging. Leveraging transfer learning, both models were initialized with pretrained weights and sub-sequently fine-tuned on the NuInsSeg dataset, which comprises over 30,000 manually segmented nuclei across 665 image patches from various human and mouse organs. This approach facilitated faster convergence and improved generalization, particularly given the limited size and high complexity of the medical dataset. The models were evaluated against key performance metrics. The experimental results reveal that YOLOv11n-seg outperforms YOLOv9c-seg with a precision of 0.87, recall of 0.84, and mAP50 of 0.89, indicating superior segmentation quality and more accurate delineation of nuclei contours. This study highlights the robust performance and efficiency of YOLOv11n-seg, demonstrating its superiority in medical image segmentation tasks, with notable advantages in both accuracy and real-time processing capabilities.