Ensemble' Boltzmann Units have Collective Computational Properties like those of Hopfield and Tank Neurons
Mark Derthick, Joe Tebelskis · Neural Information Processing Systems · 1987
There are three existing connection::;t models in which network states are assigned a computational energy. These models-Hopfield nets, Hopfield and Tank nets, and Boltzmann Machines-search for states with minimal energy. Every link in the net work can be thought of as imposing a constraint on acceptable states, and each vio lation adds to the total energy. This is convenient for the designer because constraint satisfaction problems can be mapped easily onto a network. Multiple constraints can be superposed, and those states satisfying the most constraints will have the lowest energy. Of course there is no free lunch. Constraint satisfaction problems are generally combinatorial and remain so even with a parallel implementation. Indeed, Merrick Furst (personal communication) has shown that an NP-complete problem, graph col oring, can be reduced to deciding whether a connectionist network has a state with an energy of zero (or below). Therefore designing a practical network for solving a problem requires more than simply putting the energy minima in the right places. The topography of the energy space affects the ease with which a network can find good solutions. If the problem has highly interacting constraints, there will be many local minima separated by energy barriers. There are two principal approaches to search ing these spaces: monotonic gradient descent, introduced by Hopfield [1] and refined by Hopfield and Tank [2]; and stochastic gradient descent, used by the Boltzmann Machine [3]. While the monotonic methods are not guaranteed to find the optimal solution, they generally find good solutions much faster than the Boltzmann Machine. This paper adds a refinement to the Boltzmann Machine search algorithm analogous to the Hopfield and Tank technique, allowing the user to trade off the speed of search for the quality of the solution.