Regression Model Stochastic Search via Local Orthogonalization
Ruoxi Xu · OhioLink ETD Center (Ohio Library and Information Network) · 2011
The Bayesian approach to variable selection has become increasingly popular in high dimensional problems for the reason of being able to incorporate model uncertainty into the analysis.This dissertation reviews a variety of conjugate and nonconjugate prior setups for the Bayesian model uncertainty problem with a particular focus on the point mass prior settings that allow literal exclusion of certain predictors from the model.Stochastic search algorithms for the purpose of exploring the model space, such as Markov chain Monte Carlo methods, need to be implemented carefully under point mass prior settings to ensure proper convergence and fast exploration.Motivated by the difficulty with the Gibbs sampler in making efficient moves in the model space under severe multicollinearity, this dissertation presents a locally orthogonalized Metropolis-Hastings algorithm (LOMH) for the point mass prior settings, that utilizes orthonormal rotations in the parametrization of the model to facilitate the model space exploration.This algorithm samples from the joint parameter-model space, and therefore works under both conjugate and non-conjugate prior setups.Two important variants of LOMH are also introduced in an effort to further improve its efficiency in exploring the model space.The performances of LOMH and two of its variants are illustrated through comparisons with other popular Markov chain Monte Carlo methods under conjugate and non-conjugate setups.Several simulated data It is always hard to say thank you because most of the time it means the end of something.But for the last five years, the support and help I received from my friends and colleagues has never ended.It is of great gratitude that I would like to thank them for their everlasting help and support.My primary thanks go to my advisor, Chris Hans, for his encouragement and guidance during the development of my research.This dissertation would never become a reality without his encouragement when I was deeply trapped by stumbling blocks, his hands-on guidance when I tried to learn how to write a C++ program and his insightful suggestions on my dissertation writing.His patience and support lighted my path to the completion of the graduate program like a superstar in the utter darkness and will continue to shine like a beacon for the rest of my life.I also would like to thank Steven MacEachern and Xinyi Xu for giving me an opportunity to get involved in a research project that helped me build my understanding of Bayesian modeling.Steven MacEachern has always been willing to engage in lengthy discussion about any difficult situations and has cheerfully pointed me on the right direction on more than one occasions.