Storage Capacity of the Tree-Structured Committee Machine with Discrete Weights
Robert Urbanczik · Europhysics Letters (EPL) · 1994
Gardner's method is used to calculate the storage capacity of a tree-structured committee machine with many hidden units, when its weights are constrained to take on one of 2 L equidistantly and symmetrically spaced values. Calculating the machines storage capacity from the vanishing of the replica-symmetric entropy gives for large L a capacity of (16/π) √log L which does not saturate the information-theoretic bound. This holds if the number of hidden units is much greater than 2 log L. If L exceeds a critical value, which is approximately 8 for large K , replica symmetry breaking occurs. One step of RSB gives a capacity of (8 √2/π) √log (log L ) for large L and K.