Similarities in Neuron Vectors and The Implications to CNN Inferences
Ning Lin, Xipeng Shen · NCSU Libraries Repository (North Carolina State University Libraries) · 2018
This technical report presents deep reuse, a method for speeding up CNN inferences by detecting and exploiting deep reusable computations on the fly.It empirically reveals the massive similarities among neuron vectors in activation maps, both within CNN inferences on an input and across inputs.It gives an in-depth study on how to effectively turn the similarities into beneficial computation reuse to speed up CNN inferences.The investigation covers various factors, ranging from the clustering methods for similarity detection, to clustering scopes, similarity metrics, and neuron vector granularities.The insights help create deep reuse.As an on-line method, deep reuse is easy to apply and adaptive to each CNN (compressed or not), and its input.Using no special hardware support or CNN model changes, this method speeds up inferences by 1.77-2X (up to 4.3X layer-wise) on the fly with virtually no (<0.0005)loss in accuracy.