Boosted Dense Segmentation Networks For Constrained Distributed Systems
John Brandon Graham-Knight, Abtin Djavadifar, Homayoun Najjaran, Patricia Lasserre · 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · 2021
Deployed AI applications are often heavily con-strained in computational resources; to this end, a method of producing miniature, well-performing neural networks for 2D image segmentation is devised and applied to the Severstal Steel Defect Detection and Kidney Tumor Segmentation (KiTS19) Challenges. By limiting the width of U-Net and employing a full-domain activation function, a network of aggregated weak learners is able to achieve a mean F1score within 93% of the EfficientNetB0 baseline using only 0.5% of the trainable parameters. A similar network of aggregated strong learners matches the mean F1score of the baseline on the Severstal dataset using only 10% of the trainable parameters. Gradient boosting is then applied to the weak learners, achieving a mean F1score within 98% of the strong learner network on the Severstal dataset with approximately 20% of the FLOPS; the key insight is in constraining the O(n2) relationship between network width and FLOPS. The same approach is applied to the KiTS19 dataset with good success in kidney detection. Interestingly, the method does not perform as well on the much harder to isolate tumor class, and the authors explore some possible reasons. In analyzing the impact of the full-domain activation function, the authors show that density of information is promoted by significantly reduced peaks in layer outputs and a wider range of output values. The method has significant implications in constrained deployments, as many small devices could be used to compute the overall network.