Variational inference for heteroscedastic and longitudinal regression models
Marianne Menictas · UTS ePRESS (University of Technology Sydney) · 2015
The focus of this thesis is on the development and assessment of mean field variational Bayes (MFVB), which is a fast, deterministic tool for inference in a Bayesian hierarchical model setting.We assess the performance of MFVB via the use of comprehensive comparisons against a Markov chain Monte Carlo (MCMC) benchmark.Each of the models considered are special cases of semiparametric regression.In particular, we focus on the development and assessment of the performance of MFVB for heteroscedastic and longitudinal semiparametric regression models.Generally, the new MFVB methodology performs well in its assessment of accuracy against MCMC for the semiparametric and nonparametric regression models considered in this thesis.It is also much faster and is shown to be applicable to real-time analyses.Several real data illustrations are provided.Altogether, MFVB proves to be a credible inference tool and a good alternative to MCMC, especially when analysis is hindered by time constraints.viii THEOREMS, DEFINITIONS AND RESULTS Useful vector and matrix resultsResult 1.4.11.If a and b are vectors of equal length then diag(a)b = a ⊙ b.Result 1.4.12.If A and B are matrices such that the product AB can be formed, then tr(AB) = tr(BA).Result 1.4.13.If A and B are m × n matrices, then tr(A ⊺ B) = vec(A) ⊺ vec(B).Result 1.4.14.If A, B, C, D are matrices such that the products AC and BD can be formed, then (A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD).Result 1.4.15.If A is an m × m matrix and B is an n × n matrix, then tr(A ⊗ B) = tr(A)tr(B).Result 1.4.16.If A, B, C, D are matrices such that ABCD is a square matrix, then tr(ABCD) = vec(D) ⊺ (A ⊗ C ⊺ )vec(B ⊺ ).Result 1.4.17.Let x be a random vector.Then15 1.5.MEAN FIELD VARIATIONAL BAYES Result 1.4.18.Let x be a random vector and A be a random matrix.Then E(u ⊺ Au) = {E (u)} ⊺ A {E (u)} + tr {Cov(u)A} .Result 1.4.19.Let a be constant vector, let b be a random vector, and let C be a constant matrix with the same number of rows as b.Then E(∥ a -Cb ∥ 2 ) =∥ a -CE(b) ∥ 2 +tr {CCov (b)C ⊺ } .Result 1.4.20.Let x and y be n × 1 random vectors such that x is conditioned on y.Then, Cov(y) = E {Cov(y|x)} + Cov {E(y|x)} .