Characterizing and Optimizing Transformer Inference on ARM Many-core Processor
Jiazhi Jiang, Jiangsu Du, Dan Huang, Dongsheng Li, Zheng Jiang, Yutong Lu · 2022
Transformer has experienced tremendous success and revolutionized the field of natural language processing (NLP). While GPU has become the de facto standard for deep learning computation in many cases, there are still many scenarios where using CPU for deep learning remains a prevalent choice. In particular, ARM many-core processor is emerging as a competitive candidate for HPC systems, which is promising to deploy Transformer inference.