Mitigate catastrophic forgetting for continuously learning linked open data using modularity
Lu Chen, Masayuki Murata · 2018
Nowadays, Linked Open Data is spreading year by year, and its further utilization is expected. Since the size of the data is large, Linked Open Data is attempted to learn by using neural networks. Since the data is still scaling in various region, unlike the existing neural network specialized to learn only one region, a neural network which can continuously learn wide region of knowledge is needed. However, neural network is known in its problem, catastrophic forgetting, which is to lose previously acquired skills when learning a new skill. In this research, we approach catastrophic forgetting from modularity, which is said to exist in human's brain functional network. It is true that there are many previous researches in catastrophic forgetting of neural network, however, most of the research need to know the number of learning tasks in advance, so that it is not applicable for continuously learning wide region of knowledge. The basic idea of our design approach is to design a neural network with multiple learning subsets only considering modularity without considering the number of tasks. In the evaluation, we evaluated not only for tasks learned just before, but also for tasks learned a while before. Our results show that, although, as we can expected, a neural network with high modularity can mitigate forgetting for tasks learned just before because of the low interference, a neural network with low modularity is better for the worst case when evaluating for all the tasks it learned in the past.