Game of Threads
Jose Rodrigo Sanchez Vicarte, Benjamin Schreiber, Riccardo Paccagnella, Christopher W. Fletcher · 2020
As data sizes continue to grow at an unprecedented rate, machine learning training is being forced to adopt asynchronous algorithms to maintain performance and scalability. In asynchronous training, many threads share and update the model in a racy fashion to avoid costly inter-thread synchronization.