Massively parallel mixed-signal vlsi kernel machines
Gert Cauwenberghs, Roman Genov · 2003
Recently it has been shown that a simple learning paradigm, the support vector machine (SVM), outperforms some of the most elaborately tuned expert systems and neural networks in object recognition tasks. In run-time, the SVM operates by computing a kernelbased distance between the object’s vector at the input and a set of support vectors selected from the training set, and weighting the results to produce the oracle at the output. Realtime SVM recognition of complex objects in streaming video incurs an excessive amount of computation, well beyond even the most powerful digital signal processors available today. This calls for a radically different computational paradigm to efficiently compute kernels in very large dimensions. I present a massively parallel, fine-grain distributed architecture for real-time kernel “machines” and its efficient implementation in mixed-signal VLSI technology. At the core of the externally digital architecture is a high-density, low-power analog array performing binary-binary matrix-vector multiplication, as the elementary operation in computing inner-product based kernels between presented input and stored support vectors. The threetransistor unit cell in the analog array combines a charge injection device (CID) binary multiplier and analog accumulator with embedded dynamic random-access memory (DRAM). I present various schemes to obtain precise digital results from the internal analog computation in a distributed, parallel fashion, using analog-to-digital quantization of partial binary-binary products computed over the array. High output resolution is achieved with low complexity quantizers by oversampling in the input binary representation combined with delta-sigma modulated quantization at the output. In addition, stochastic encoding of the digital inputs relaxes the precision requirements of the quantizers by the square root of the vector dimension owing to the Central Limit in the accumulation of binary terms in the inner-product.