Fully-Automatic Bayesian Piecewise Sparse Linear Models
Riki Eto, Ryohei Fujimaki, Satoshi Morinaga, Hiroshi Tamano · 2015
Piecewise linear models (PLMs) have been widely used in many enterprise machine learning problems, which assign linear ex-perts to individual partitions on feature spaces and express whole models as patches of local experts. This paper addresses si-multaneous model selection issues of PLMs; partition structure determination and feature selection of individual experts. Our contri-butions are mainly three-fold. First, we ex-tend factorized asymptotic Bayesian (FAB) inference for hierarchical mixtures of experts (probabilistic PLMs). FAB inference offers penalty terms w.r.t. partition and expert complexities, and enable us to resolve the model selection issue. Second, we propose posterior optimization which signicantly im-proves predictive accuracy. Roughly speak-ing, our new posterior optimization miti-gates accuracy degradation due to a gap be-tween marginal log-likelihood maximization and predictive accuracy. Third, we present an application of energy demand forecasting as well as benchmark comparisons. The ex-periments show our capability of acquiring compact and highly-accurate models. 1