Inter- and intra-observer variability in radiologists' assessment of mass similarity on mammograms
Berkman Sahiner, Lubomir M. Hadjiiski, Heang‐Ping Chan, Jing Cui, Chintana Paramagul, Alexis V. Nees, Mark A. Helvie · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2009
The purpose of this study was to compare the performances of two recently-developed image retrieval methods for mammographic masses, and to investigate the inter- and intra-observer variability in radiologists' assessment of mass similarity. Method 1 retrieved masses that are similar to a query mass from a reference library based on radiologists' margin and shape descriptions and the mass size. Method 2 used computer-extracted features. Two MQSA radiologists participated in an observer study in which they rated the similarity between 100 query masses and the retrieved lesions based on margins, shape, and size. For each query mass, three masses retrieved using Method 1 and three masses retrieved using Method 2 were displayed in random order using a graphical user interface. A nine-point similarity rating scale was used, with a rating of 1 indicating lowest similarity. Each radiologist repeated the readings twice, separated by more than three months, so that intra-observer variability could be studied. Averaged over the two radiologists, two readings, and all masses, the mean similarity ratings were 5.59 and 5.57 for Methods 1 and 2, respectively. The difference between the two methods did not reach significance (p>0.20) for either radiologist. The intra-observer variability was significantly lower than the inter-observer variability, which may indicate that each radiologist may have their image similarity criteria, and the criteria may vary from radiologist to radiologist. The understanding of the trends in radiologists' assessment of mass similarity may guide the development of decision support systems that make use of mass similarity to aid radiologists in mammographic interpretation.