An optimizing compiler for ONNX models on heterogeneous systems
Yuanjing Shi · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · 2020
In order to build, train, and deploy deep learning models for modern data-driven applications, programs need to be executed on top of specialized heterogeneous systems for better performance. However, programming on those heterogeneous systems remains a fundamental challenge in terms of the interoperability issue between high-level deep learning frameworks and the programmability issue between different low-level heterogeneous systems. In this work, we propose a portable and highly optimizing compiler for neural network models, which is based on an open format - ONNX of deep learning models, running on heterogeneous systems. It consists of a front-end and a back-end to address those above issues. The goal of this neural network compiler is also to map high-level neural network models to low-level executable programs. We evaluate this work with several deep learning neural network models and our neural network compiler is able to outperform ONNX runtime by up to 3.15x and Keras by up to 4.37x on certain workloads.