Information-theoretic grounding of finite automata in neural systems
Thomas Wennekers, Nihat Ay · 2002
We introduce a measure “stochastic interaction” that captures spatial and temporal signal properties in recurrent systems. The measure quantifies the Kullback-Leibler divergence of a Markov chain from a product of split chains for the single units. Maximization of stochastic interaction, also called “Temporal Infomax”, is shown to induce almost deterministic dynamical systems for unconstrained Markov chains. If part of the units are clamped to prescribed stochastic processes providing external input, Temporal Infomax leads to finite automata, either completely deterministic or at most weakly nondeterministic. This way, computational capabilities may arise in neural systems. 84.35.+i, 87.19.La, 02.50.Ga Typeset using REVTEX