The Consolidation of Neural Network Task Knowledge.
Daniel Silver, Peter McCracken · International Conference on Machine Learning and Applications · 2003
A fundamental question of any life-long learning system is addressed: How can task knowledge be consolidated within a long-term domain knowledge structure for efficient storage and for more efficient and effective transfer of that knowledge when learning a new task. A review of relevant background material on knowledge based inductive learning and the sequential transfer of task knowledge using multiple task learning (MTL) neural networks is presented A theory of task knowledge consolidation is proposed that uses a large MTL network as the domain knowledge structure and task rehearsal as a method of overcoming the catastrophic forgetting problem. The theory is tested on a synthetic domain of seven tasks and it is shown that task knowledge can be sequentially consolidated within a domain knowledge MTL network both effectively and efficiently.