Parallel intelligent computing: development and challenges
凯 卢, 志权 赖, 笙维 李, 炜杰 柳, 可适 葛, 锡城 卢, 东升 李 · Scientia Sinica Informationis · 2023
The development of artificial intelligence technology with the use of deep learning has gained momentum in recent years, resulting in a considerable increase in the scale of deep learning models and training data, and high-performance computing and artificial intelligence technologies have been continuously and deeply integrated. Parallel intelligent training has become the main method for the efficient training of large-scale deep learning models. In this work, we evaluate the basic methods and key technologies of parallel intelligent training, outline the development status of the parallel intelligent training framework, summarize the challenges and trends in developing the parallel intelligent technology and framework, and propose development schemes for a Merak parallel intelligent training framework.