A systematic evaluation of learning rate policies in training CNNs for brain tumor segmentation
Syed Talha Bukhari, Hassan Mohy‐ud‐Din · Physics in Medicine and Biology · 2021
Abstract Convolutional neural networks (CNNs) have recently emerged as a powerful approach for automatic segmentation of brain tumor subregions on 3D multi-parametric MRI scans. Learning rate is a crucial hyperparameter in the training of CNNs, impacting the performance of the learned model. Different learning rate policies trace unique trajectories in the optimization landscape that converge to local minima with varying generalization properties. In this work, we empirically evaluated nine learning rate policy-optimizer pairs with two state-of-the-art architectures, namely 2D slice-based U-Net and 3D DeepMedicRes, on an augmented brain tumor dataset of 534 subjects. Segmentation performance was quantified in terms of Dice similarity coefficient and Hausdorff distance metrics. The policies were ranked based on the final ranking score (FRS) employed by the BraTS challenge, with the statistical significance of the rankings evaluated by random permutation test. For 2D slice-based U-Net architecture, an overall ranking of learning rate policies showed that the polynomial decay policy with Adam optimizer significantly outperformed other policies for the task of individual and hierarchical segmentation of tumor subregions ( p 0.3). These findings were also validated on the BraTS 2019 Validation dataset which comprised of an additional 125 subjects.