Neural ODE Machine Learning Method with Embedded Numerical Method
Andrey Televnoy, Sergei Ivanov, Tatiana Victorovna Zudilova, Tatiana Voitiuk · 2021
In the study, the authors examine the application of the Neural ODE machine learning method to reduce the number of neural network layers in order to minimize the problem of fading gradients. It is proposed to implement in Neural ODE a new scheme of the numerical Runge-Kutta method developed by the authors. The mathematical substantiation of the proposed numerical method, which guarantees the accuracy of the third order, is presented in detail. A comparative analysis of the training and computation time for Neural ODE with other machine learning methods using a computational experiment. The results of the experiment show that the proposed modification of the Neural ODE can reduce the computation time and the network training time.