Self-organizing neural networks based on gaussian mixture model for pdf estimation and pattern classification
Sukhan Lee, S. Shimoji · 1994
This work proposes three new neural network models in pattern processing for (1) PDF (probability density function) estimation, (2) pattern classification, and (3) feature extraction, while the major emphasis is placed on the PDF estimation network. The distinctive features of the proposed PDF estimation method are as follows: (1) PDF of a class is modeled as being composed of a number of Gaussian kernels called subclasses (Gaussian Mixture Model). During the network training, given class samples are decomposed probabilistically into subclass samples based on the current subclass PDFs, and the subclass PDFs are updated iteratively so that the discrepancy between the current and the actual subclass PDFs are reduced. (2) Subclasses are automatically recruited to compensate the inaccuracy of the PDF estimation, which is caused by the lack of the number of subclasses and/or the occurrence of local minima states (Self-Organization). The local minima states are detected by applying Chi-square tests to individual subclasses. The proposed method provides the semi-parametric estimation of an arbitrary form of PDFs, which allows the network to avoid local minima. In addition, it is shown that the parameters obtained by the network training are equivalent to the maximum likelihood estimation. As an application of the PDF estimation network, a pattern classification network is designed based on Bayesian criterion, which is known as the theoretically optimal classification rule. Furthermore a new method of feature extraction is proposed. Unlike conventional feature extraction systems which process patterns based on the variance of the distribution, the proposed method promotes clustering of features to comply with the classification process based on the Gaussian Mixture Model. Simulations demonstrate the superiority of the PDF estimation network, the feature extraction system and the classification system.