Differential Evolution Neural Network Optimization with Individual Dependent Mechanism
Naoya Ikushima, Keiko Ono, Yuya Maeda, Erina Makihara, Yoshiko Hanada · 2021
With the increase of scenes where Neural Networks are used as a classifier, the expectation of the classifying accuracy for the network has risen. To improve classifying accuracy, Differential Evolution (DE) has been applied as an optimization method for Neural Networks. Compared to other DE methods, Differential Evolution with an Individual-Dependent Mechanism (IDE) takes in account of the differences between the fitness value of individuals. As a result, IDE has better results in terms of accuracy. Therefore in this paper, a Neural Network optimizer using IDE is proposed for further Neural Network improvement. Moreover, while most optimizers would use the loss function as the fitness value in DE, a method using accuracy is proposed because DE does not require the activation function to be differentiable. Experiments using the proposed framework has been conducted on classification problems, and comparative studies with other self-adaptive mutations and traditional optimizing methods have been performed. Experimental results showed that the proposed method outperformed other conventional methods in terms of accuracy.