Statistical mechanics of lossy compression for nonmonotonic multilayer perceptrons
Florent Cousseau, Kazushi Mimura, Toshiaki Omori, Masato Okada · Physical Review E · 2008
A lossy data compression scheme for uniformly biased Boolean messages is investigated via statistical mechanics techniques. We utilize a treelike committee machine (committee tree) and a treelike parity machine (parity tree) whose transfer functions are nonmonotonic. The scheme performance at the infinite code length limit is analyzed using the replica method. Both committee and parity treelike networks are shown to saturate the Shannon bound. The Almeida-Thouless stability of the replica symmetric solution is analyzed, and the tuning of the nonmonotonic transfer function is also discussed.