Using information entropy bounds to design VLSI friendly neural networks
Sorin Drăghici · 2002
This paper presents a method for calculating the minimal size of a VLSI optimal network for a given problem. A VLSI optimal network is a network using integer weights in a given range [-p,p] and units with a small constant fan-in. The value p is a small integer which is calculated from the problem parameters such that the problem is guaranteed to have a solution. It is shown that the number of weights can be lower bounded in the worst case by the expression: m(n+1)[nlog(/sub dmin///sup dmx/)+(n-1)log(/sub dmin///sup 1/+1)+ 1/2 log(n-1)-/sub 12.(n-1)//sup loge/+c] where d/sub min/ is the minimum distance between patterns of opposite classes, d/sub max/ is the maximum distance between any patterns, m is the number of patterns in the largest class, n is the number of dimensions and c is a constant. The methodology is tested on various problems using a limited precision integer weights constructive algorithm.