Gated competitive systems for unsupervised segmentation and modeling of piecewise stationary signals
Craig Fancourt, José Carlos Príncipe · University of Florida Digital Collections (University of Florida) · 1998
The major goal of this dissertation is to present a new paradigm for the unsupervised competitive segmentation and modeling of signals into stationary segments, and to elucidate and develop several specific algorithms that fall within the paradigm. The identification of the segments is accomplished by several experts which act in parallel on a local section of the signal at a time. The experts can utilize any unsupervised learning algorithm to which one can ascribe a performance measure, the two most common of which are the tasks of prediction and auto-association. The experts learn in the usual way except that the local data are weighted based on how well the experts have performed in the past. This information is provided by the gate, which analyzes the performance of the experts and then ascribes to each a relative measure of the validity of each expert to that part of the data. The cumulative effect of these gate assignments is to segment the data. A gate can be either input or output based. An input based gate attempts to learn to predict the validity of the experts from the input only, thus eventually eliminating the need for performance information, while an output based gate always analyzes the performance of the experts. We use principal component analysis (PCA) experts, which requires finding the probability density function for auto-association, and then show that the resulting learning equations closely parallel those of a single PCA expert. When applied to time series, PCA becomes temporal PCA. We show that the discrete prolate spheroid wave functions form a natural basis for the eigenfunctions in the frequency domain, and that this basis explains most of the properties of temporal PCA. Finally, we show that momentum learning is equivalent to a recursive mean square error estimator, and then use this estimator to add memory to the gate, which improves segmentation and learning by giving greater validity to experts that have performed well in the recent past.