Probabilistic approach to unsupervised representation learning in dynamic environments
Jun-ichiro Hirayama · Institutional Repositories DataBase (IRDB) · 2007
In the latter part, I investigate an online feature extraction problem, particularly focusing on such situations that the environment is not stationary but dynamically changing, sometimes even abruptly.In this difficult non-stationary context, an appropriate control of adaptability/stability of a learning model is the key for rapid adaptation to the environmental changes.Focusing on feature extraction task by means of the probabilistic PCA (PPCA), I propose two online learning schemes that have such a character, based on the previously-proposed online variational Bayes (VB) method: One is based on an explicit formulation of probabilistic novelty detection by a mixture of PPCA model, the other is a principled approach using the hierarchical Bayes method based on a new interpretation of the online VB presented in this thesis.I demonstrate their availabilities in simulation experiments.In addition, I also discuss the biological implication of the proposed learning models, especially with hypothesizing their possible realization in brain.While the methods proposed in this thesis have been developed as general statistical techniques to analyze and process stochastic data that inherently have dynamic natures, their high performances demonstrated in the simulation experiments indicate the future availabilities for specific real-world problems.Thus, I finally discuss the potential applications of the proposed methods with also discussing open issues remained for future developments of these methods.