Stochastic Gradient Descent as Approximate Bayesian Inference
Stephan Mandt, Matthew D. Hoffman, David M. Blei · Journal of Machine Learning Research · 2017
Stochastic Gradient Descent with a constant learning rate (constant SGD) simulates a Markov chain with a stationary distribution. With this perspective, we derive several new results. (1) We show t...