A 28nm 77.35TOPS/W Similar Vectors Traceable Transformer Processor with Principal-Component-Prior Speculating and Dynamic Bit-wise Stationary Computing
Yang Wang, Yubin Qin, Dazheng Deng, Xiaolong Yang, Zhiren Zhao, Ruiqi Guo, Zhiheng Yue, Leibo Liu, Shaojun Wei, Yang Hu, Shouyi Yin · 2023
This paper proposes an energy-efficient Transformer processor exploiting dynamic similarity in global attention computing. It has three features: 1) A principal-component-prior speculation unit (PCSU) removes 28.4% of redundant computations. 2) A similar-vector tracked computing engine (STCE) saves 42.2% of multiplications. 3) A bit-wise stationary processing element (BSPE) reduces multiplication energy by $1.47\times$. The proposed processor achieves a peak energy efficiency of 77.35TOPS/W. It reduces energy by $2.81\times$ and offers $3.71\times$ speedup compared with the state-of-the-art Transformer processor.