REX-SC: Range-Extended Stochastic Computing Accumulation for Neural Network Acceleration
Tianmu Li, Wojciech Romaszkan, Sudhakar Pamarti, Puneet Gupta · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · 2023
Deep learning has grown in capability and size in recent years, prompting research on alternative computing methods to cope with the increased compute cost. Stochastic computing (SC) promises higher compute efficiency with its compact compute units, but accuracy issues have prevented wide adoption, and accuracy-improving techniques have sacrificed runtime or training performance. In this work, we propose extended range SC—Range-Extended SC Accumulation to deal with the accuracy issues of SC. By modifying the functionality of OR-based SC accumulation, we increase SC computation accuracy without sacrificing the performance benefits. Our approach achieves a$2\times $reduction in stream length for the same accuracy compared to SC with OR-based accumulation and an up to$3.6\times $improvement in energy compared to SC with binary addition. With proper modeling, our approach improves training performance for SC-based neural networks and makes training SC models practical for large datasets like ImageNet.