AIC-Bench: Workload Selection Methodology for Benchmarking AI Chips
Zhenyu Quan, Xiaoming Chen, Yinhe Han · 2022
Artificial intelligence (AI) chips are gaining tremendous attention due to their successes in accelerating deep learning algorithms. AI chips need standard benchmarks to drive the advancements in their software and hardware systems. However, the deep learning workloads in most existing AI benchmarks are fixed and have similar computation characteristics. Therefore, we introduce AIC-Bench, a deep learning workload selection methodology for comparing the performance of AI chips. The core of the proposed method is to select a number of representative deep learning workloads in terms of computational and memory access characteristics to represent other workloads fully. As a case study, we select eight workloads from a workload candidate pool with the proposed method. Using the eight selected workloads, we evaluate the performance of CPU, GPU, and MLU. We also compare the energy efficiency between GPU and MLU. Based on the evaluation result, suggestions for performance improvements of these AI chips are put forward.