Optimization for problem classes-neural networks that learn to learn
Michael Hüsken, Jens Gayko, Bernhard Sendhoff · 2002
The main focus of the optimization of artificial neural networks has been the design of a problem dependent network structure in order to reduce the model complexity and to minimize the model error. Driven by a concrete application we identify in this paper another desirable property of neural networks-the ability of the network to efficiently solve related problems denoted as a class of problems. In a more theoretical framework the aim is to develop neural networks for adaptability-networks that learn (during evolution) to learn (during operation). Evolutionary algorithms have turned out to be a robust method for the optimization of neural networks. As this process is time consuming, it is therefore also from the perspective of efficiency desirable to design structures that are applicable to many related problems. In this paper, two different approaches to solve this problem are studied, called ensemble method and generation method. We empirically show that an averaged Lamarckian inheritance seems to be the most efficient way to optimize networks for problem classes, both for artificial regression problems as well as for real-world system state diagnosis problems.