Observer Performance Factors Influencing Breast Cancer Detection in Mammography
Wasfi I. Suleiman · The Sydney eScholarship Repository (The University of Sydney) · 2016
Aims: Errors and variability in mammography interpretation is well established, however factors responsible for these are unclear. Therefore, it is important to explore parameters that impact upon performance as well as ways of improving performance and reducing inter-reader variability. This thesis aims to 1. examine whether work practices and reader characteristics such as number of cases read per year and experience impact upon performance by comparing radiologists from Australia and the US; 2. compare readers’ performance in detecting Architectural Distortion (AD) compared with other breast cancer types using digital mammography (DM); 3. assess whether radiologists who regularly undertake the Breast Reader Assessment Strategy (BREAST) demonstrate improvement in performance over time. Materials and Methods: The work is divided into three phases under two institutional review board approvals. Phase 1 compared the performance of Australian and U.S. radiologists and the impact of cases read per year and experience on performance. In phase 2, the same groups of readers as in the first study were used to compare radiologists’ performance in detecting AD compared to non-AD. In the third phase, 14 Royal Australian and New Zealand College of Radiology (RANZCR) radiologists independently assessed a year-specific BREAST mammographic test-set in each of the years, 2011, 2012 and 2013. The mean sensitivity, specificity, location sensitivity, JAFROC FOM and inferred ROC AUC were calculated and compared. Results: In phase 1, when experience and the number of mammograms read per year were taken into account, the Australian radiologists sampled showed significantly higher sensitivity and location sensitivity (p ≤ 0.001). JAFROC (FOM) and inferred ROC (AUC) analysis showed no difference between the overall performances of the two countries. ROC (AUC) and location sensitivity were higher for the Australian radiologists who read the most cases per year. In phase 2, significantly lower location sensitivity (p ≤ 0.0001) and higher false negative (FN) values (p ≤ 0.0001) were noted for AD cases compared with non-AD cases for all reader groups. In phase 3, significant increases were noted in mean sensitivity (p = 0.01), specificity (p = 0.01), location sensitivity (p = 0.001), JAFROC FOM (p = 0.001) and ROC AUC (p = 0.001) between 2011 and 2012. There were also increases in mean sensitivity (p = 0.002), specificity (p = 0.001), location sensitivity (p = 0.001), JAFROC FOM (p = 0.001) and ROC AUC (p = 0.001) between 2011 and 2013. Conclusion The findings show that an increased number of mammographic cases read per year increases readers’ sensitivity, and that AD detection remains difficult to diagnose. Improvements were shown in the diagnostic performance of radiologists undertaking the BREAST test-set reading programme over a three-year period. Finally, it has demonstrated that carefully constructed test-sets for education can improve lesion detection with DM in test-set conditions.