Predicting Malignancy and Benign Thyroid Nodule Using Multi-Scale Feature Fusion and Deep Learning
Xinyi Wei, Siwei Zhang, Qi Qi, Hao Fu, Taorong Qiu, Aiyun Zhou · Pattern Recognition and Image Analysis · 2021
Abstract Nowadays, thyroid ultrasound examination faces some problems such as weak effective feature information, plentiful noise, and small samples. Our research aims at helping doctors making decision more accurately and quickly to identify the characteristics of patients’ thyroid nodules based on ultrasound images. Firstly, after pre-processing ultrasound images of thyroid nodules, a noise reduction method is proposed by using weighted adaptive gamma correction which can effectively suppress the generation of noise and improve the global information contrast ratio. Secondly, fine-tuning transfer learning to pre-train ResNet-18 convolutional neural network is used to solve over-fitting under small samples. Thirdly, an adaptive threshold Local Ternary Pattern algorithm is proposed to extract local texture features of the ultrasound images in order to enhance the classification performance. Finally, a multi-scale feature fusion approach, which combines the local texture features and the deep features (the global texture features) automatically extracted by convolutional layers, is carried out by following a second fine-tuning training in ResNet-18 convolutional neural network based on the multi-scale joint features. The test results show: (1) the improved Adaptive Threshold Local Ternary Pattern algorithm demonstrates better performance than other algorithms in extracting texture features on the experimental thyroid nodule dataset, which has fewer misclassified samples and can better describe the texture information of the ultrasound image nodules. (2) The classification accuracy is significantly promoted in the given real test set based on the improving ResNet-18 convolutional neural network by using the proposed multi-scale feature fusion approach.