A Robust Self-Constructing Normalized Gaussian Network for Online Machine Learning
Jana Cathrin Backhus · Hokkaido University Collection of Scholarly and Academic Papers (Hokkaido University) · 2017
In this thesis, I aim to improve the robustness and applicability of Normalized Gaussian networks (NGnet) in the context of online machine learning tasks. A challenging problem in online machine learning is the limited domain knowledge provoked by restricted prior knowledge, while additional information are received only sequentially over time. The limited domain knowledge makes the application of artificial neural networks (ANN) more difficult, and major challenges include negative interference and the selection of an accurate model complexity. In this thesis, I consider these challenges in regard to the NGnet, which belongs to a group of ANNs that possess a certain grade of robustness against negative interference due to the local properties of their network architecture. Yet, further improvements of robustness are necessary in regard to the ANN’s learning algorithm and model complexity selection. A recently proposed learning algorithm with localized forgetting provides robustness against negative interference, but it is not applicable over the full numerical range of an implied discount factor. Also, dynamic model selection was yet to be considered. Therefore, I revise the localized forgetting approach and adapt dynamic model selection to it in a self-constructing manner. Dynamic model selection has been considered for an earlier learning algorithm of the NGnet with global forgetting, which however shows a non-robust behavior in negative interference prone environments. Then, I propose localization of some of the model selection mechanisms to improve their robustness and add a new merge manipulation to deal with model redundancies. The effectiveness of the proposed method is compared with earlier learning approaches of the NGnet for several experiments. The proposed method possesses robust and favorable performance in the different tested learning environments, making it the better alternative when applied to online learning tasks with proneness to negative interference.