Breast mass image retrieval based on multimodality similarity estimation
Chisako Muramatsu, Mikinao Oiwa, Tomonori Kawasaki, Hiroshi Fujita · 2020
Retrieval of similar cases can help radiologists in efficient diagnosis, treatment planning, and preparation of reports for new cases. In this study, similarities of pairs of lesions were estimated using convolutional neural networks with subjective similarity data. The network was trained with pairs of mammograms (MG), pairs of ultrasound images (US), and both as input data and the corresponding similarity ratings by expert radiologists as teacher data. Based on the estimated similarity, the cases with the highest similarities were retrieved for a test case. The precisions of selecting pathology-matched relevant cases were compared for the networks using different input data. In this study, rather a simple network architecture, which takes a pair or pairs of input images and has one regression output layer corresponding to the similarity, provided higher precisions. The precisions using mammograms, ultrasound images, and both modalities were 0.72, 0.68, and 0.80, respectively. The highest precision was obtained by the use of one network with multimodality image inputs than combining the outputs by two separate networks for MG and US data. Relatively high precision indicates that the presentation of reference images can be useful for assisting breast cancer diagnosis.