BK.Synapse
Dinh Viet Sang, Phan Ngoc Lan · 2019
Training neural networks efficiently is a thoroughly-researched topic that plays an important role in their adoption. Major advancements have been made, including the use of multiple nodes to further decrease training time. However, training at scale usually means adding on multiple layers of complex deployment logic and parallelization concerns, distracting researchers from the core of their algorithms. This paper presents a framework called BK.Synapse that can facilitate distributed training while maintaining clarity, simplicity, and user-friendliness. The design is modular, allowing flexible and easy deployment on a variety of hardware specifications. The framework is benchmarked in a case study: training a neural network for an object detection problem. Our results show a good amount of improvements over conventional training, with very few modifications to the existing codebase. The resulting model also performs relatively well upon further testing.