Hierarchical mixtures-of-experts for generalized linear models: some results on denseness and consistency.
Wenxin Jiang, Martin A. Tanner · 1999
We investigate a class of hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form /(ff + x T fi) are mixed. Here /(\\Delta) is the inverse link function. Suppose the true response y follows an exponential family regression model with mean function belonging to a class of smooth functions of the form /(h(x)) where h(\\Delta) 2 W 1 2;K0 (a Sobolev class over [0; 1] s ). It is shown that the HME mean functions can approximate the true mean function, at a rate of O(m \\Gamma2=s ) in L p norm. Moreover, the HME probability density functions can approximate the true density, at a rate of O(m \\Gamma2=s ) in Hellinger distance, and at a rate of O(m \\Gamma4=s ) in Kullback-Leibler divergence. These rates can be achieved within the family of HME structures with a tree of binary splits, or within the family of structures with a single layer of experts. Here s is the dimension of the ...