A variational framework for local learning with probabilistic latent representations
Cabrel Teguemne Fokam, Khaleelulla Khan Nazeer, Christian Mayr, Anand Subramoney, David Kappel · 2025
We introduce a novel method for distributed learning by dividing deep neural networks into blocks and incorporating feedback networks to propagate target information backwards, enabling auxiliary local losses.Forward and backward propagation operate in parallel with independent weights, addressing locking and weight transport problems.Our approach is rooted in a statistical view of training, treating block output activations as parameters of probability distributions to measure alignment between forward and backward passes.Error backpropagation is then performed locally within blocks, hence block-local learning.Preliminary results across tasks and architectures showcase state-of-the-art performance, establishing a principled framework for asynchronous distributed learning.* CTF and KKN are