Accelerating Monte Carlo methods for Bayesian inference in dynamical models
Johan Dahlin · Linköping studies in science and technology. Dissertations · 2016
Making decisions and predictions from noisy observations are two important and challenging problems in many areas of society.Some examples of applications are recommendation systems for online shopping and streaming services, connecting genes with certain diseases and modelling climate change.In this thesis, we make use of Bayesian statistics to construct probabilistic models given prior information and historical data, which can be used for decision support and predictions.The main obstacle with this approach is that it often results in mathematical problems lacking analytical solutions.To cope with this, we make use of statistical simulation algorithms known as Monte Carlo methods to approximate the intractable solution.These methods enjoy well-understood statistical properties but are often computational prohibitive to employ.The main contribution of this thesis is the exploration of different strategies for accelerating inference methods based on sequential Monte Carlo ( smc) and Markov chain Monte Carlo ( mcmc).That is, strategies for reducing the computational effort while keeping or improving the accuracy.A major part of the thesis is devoted to proposing such strategies for the mcmc method known as the particle Metropolis-Hastings ( pmh) algorithm.We investigate two strategies: (i) introducing estimates of the gradient and Hessian of the target to better tailor the algorithm to the problem and (ii) introducing a positive correlation between the point-wise estimates of the target.Furthermore, we propose an algorithm based on the combination of s m c and Gaussian process optimisation, which can provide reasonable estimates of the posterior but with a significant decrease in computational effort compared with pmh.Moreover, we explore the use of sparseness priors for approximate inference in over-parametrised mixed effects models and autoregressive processes.This can potentially be a practical strategy for inference in the big data era.Finally, we propose a general method for increasing the accuracy of the parameter estimates in non-linear state space models by applying a designed input signal.v x Acknowledgments myself and for all the other p h d students.Furthermore, I gratefully acknowledge the financial support from the projects Learning of complex dynamical systems (Contract number: 637-2014-466) and Probabilistic modeling of dynamical systems (Contract number: 621-2013-5524) and cadics, a Linnaeus Center, all funded by the Swedish Research Council.I would also like to acknowledge Dr. Henrik Tidefelt and Dr. Gustaf Hendeby for constructing and maintaining the L A T E X-template in which this thesis is (partially) written.Another aspect of the atmosphere at work is all my wonderful colleagues.My room mate from the early years Michael Roth was always there to discuss work and to keep my streak of perfectionism in check.My friendships with Jonas Linder and Manon Kok have also meant a lot to me.We joined the group at the same time and have spent many hours together both at work and during our spare time.Thank you for your positive attitudes and for all the fun times exploring Beijing, Cape Town, Vancouver, Varberg and France together.Furthermore, thank you Sina Khoshfetrat Pakazad for arranging all the nice b b qs and for always being up for discussing politics.It is also important to get some fresh air and see the sun during the long days at work.Hanna Nyqvist has been my loyal companion during our many lunch walks together.Oskar Ljungqvist recently joined the group but has quickly become a good friend.Thank you for all our lunch runs (jogs) together, for encouraging and inspiring my running and for all the nice discussions!Furthermore, I would like to thank my remaining friends and colleagues at the group.Especially, (without any specific ordering) Christian Andersson Naesseth, Christian Lyzell,