Bayesian growing and pruning strategies for map-optimal estimation of Gaussian mixture models
Daniel W. McMichael · 1995
Real-time learning requires on-line complexity estimation. Expectation-maximisation #EM# and sampling techniques arepresented that enable simultaneous estimation of the complexity and continuous parameters of Gaussian mixture models #GMMs# which can be used for density estimation, classi#cation and feature extraction. The solution is a maximum a posteriori probability #MAP# estimator that is convergent for #xed data and adaptive with accruing data. Issues resolved include estimating the priors for element covariances, means and weights and calculating the local integrated likelihood #evidence# of the solution. The EM algorithm for MAP estimation of GMM parameters is established and extended to include complexity estimation #ie. iterative pruning#. The EMS algorithm is introduced which incorporates a sampling stage that enables iterative growth of the GMM. Early trials involving speech data indicate that the likelihood of hidden Markov speech models can be very substantially increased u...