Strategies for Graph Optimization in Deep Learning Compilers
Yijian Zheng · 2024
This manuscript presents an exhaustive survey of graph optimization strategies in the realm of deep learning compilers, elucidating the pivotal evolution of artificial intelligence compilers. These compilers play an integral role in harmonizing software and hardware capabilities to enhance the efficiency of deep learning processes. The paper meticulously delves into a variety of front-end optimization techniques. These techniques are instrumental in refining the structure of computational graphs, thereby significantly bolstering the efficiency of neural network training and inference phases. Additionally, the paper highlights the escalating demands in computational performance, memory requirements, and hardware synchronization in the context of deep learning, outlining fertile avenues for future research. It underscores the complexity of managing increased computational loads and the intricate interplay between various hardware components. The discussion extends to the challenges and potential strategies for optimizing resource allocation and data flow within these systems.