Stochastic Bitstream-based Reservoir Computing with Feedback
David Verstraeten · 2005
Reservoir Computing is a recent pattern recognition technique that combines temporal processing capabilities with fast learning rates and excellent convergence properties. The system consists of two parts: a recurrently connected network of simple nodes (e.g. neurons) called the reservoir, and a so-called readout function which can be any traditional statistical technique and which computes the actual output. The choice of the node with which we build the reservoir is very broad, and in this case we use stochastich bitstream neurons. Classical sigmoidal neurons perform a weighted sum of their inputs, followed by a non-linearity. This uses a lot of additions and multiplications, which is not hardware efficient at all. Stochastic bitstream neurons circumvent this problem by communicating through stochastic bitstreams instead of analog values, which transforms additions and multiplications to simple bitwise operations, thus allowing an efficient hardware implementation. For this publication we built an RC-system using these types of neurons. Several proof of concept experiments were performed.