Bayesian High-dimensional Linear Regression with Sparse Projection-posterior
Samhita Pal, Subhashis Ghoshal · Bayesian Analysis · 2025
We consider a novel Bayesian approach to estimation, uncertainty quantification, and variable selection for a high-dimensional linear regression model under sparsity. The number of predictors can be nearly exponentially large relative to the sample size. First, we place a conjugate normal prior, disregarding sparsity. For inference, instead of the multivariate normal posterior, we use the posterior induced by a sparsifying map built as the sum of squares of deviations plus an ℓ1-penalty on the vector. We show that the resulting sparse projection-posterior distribution contracts around the true value of the parameter at the optimal rate adapted to its sparsity. We show that the true sparsity structure gets a large sparse projection-posterior probability. We further show that an appropriately recentred credible ball has the correct asymptotic frequentist coverage. Moreover, our method requires only summary data and can therefore operate in a distributed computing setup, respecting data privacy. We conduct a comprehensive simulation study under a variety of settings and found that the proposed method performs well for finite sample sizes. We apply the method to several real datasets, including the Alzheimer’s Disease Neuroimaging Initiative (ADNI) data. The method is implemented in the R-package sparseProj, and all computations have been conducted using this package.