A Checkpoint/Restart Scheme for CUDA Programs with Complex Computation States
Hai Jiang, Yulu Zhang, Jeff Jennes, Kuan‐Ching Li · The International journal of networked and distributed computing · 2013
Checkpoint/restart has been an effective mechanism to achieve fault tolerance for many long-running scientific applications.The common approach is to save computation states in memory and secondary storage for execution resumption.However, as the GPU plays a much bigger role in high performance computing, there is no effective checkpoint/restart scheme yet due to the difficulty of the GPU computation state handling.This paper proposes an application-level checkpoint/restart scheme to save and restore GPU computation states in annotated user programs.A pre-compiler and run-time support module are developed to construct and save states in CPU system memory dynamically, whereas secondary storage can be utilized for scalability and long-term fault tolerance.CUDA programs with complicated computation states are supported.State-related variables dissipated in various memory units are collected.Both stack and heap are duplicated at application level for state construction.Experimental results have demonstrated the effectiveness of the proposed scheme.