A Hierarchical Incremental Learning Approach to Task Decomposition
Sheng-Uei Guan, Peng Li · Journal of Intelligent Systems · 2002
In this paper, we propose a new task decomposition approachhierarchical incremental class learning (HICL).In this approach, a Κ -class problem is divided into Κ sub-problems.The sub-problems are learnt sequentially in a hierarchical structure with Κ sub-networks.Each subnetwork takes the output from the sub-network immediately below it as well as the original input as its input.The output from each sub-network contains one more class than the sub-network immediately below it, and this output is fed into the sub-network above it.It not only reduces harmful interference among hidden layers, but also facilitates information transfer between classes during training.The later sub-networks can obtain learnt information from the earlier sub-networks.We also proposed two ordering algorithms -Minimal-Side-Effect-First ordering method based on Class Decomposition Error (MSEF-CDE) and Minimal Side-Effect Ordering based on Fisher's Linear Discriminant (MSEF-FLD) to determine the hierarchical relationship between the sub-networks.The proposed HICL approach shows smaller regression error and classification error than classical decomposition approaches.