Neural Ordinary Differential Equations based System Identification for Reinforcement Learning with Provable Guarantees
Théo Rutschke, Mayank Shekhar JHA, Hugues Garnier · 2025
This paper investigates nonlinear identification for control learning with provable guarantees. A novel approach is proposed for Model-Based Reinforcement Learning (MBRL) where Neural Ordinary Differential Equation (NODE) based nonlinear system identification in continuous time is integrated within Policy Iteration (PI) based Reinforcement Learning. To that end, first, a continuous-time NODE model is identified from measured data, which is then leveraged to learn an optimal controller using off-policy PI. Rigorous proofs are developed to guarantee boundedness of parameter as well as prediction errors. The identified NODE model enables admissible initialization of the PI algorithm through a Quadratic Program (QP) under NODE-based Control Lyapunov Function (CLF) constraints, leading to guaranteed admissibility and stability at the initialization phase of control learning. During the exploration phase, closed-loop stability is maintained by enforcing NODE-based Input-to-State Stable CLF (ISS-CLF) constraints. The resulting controller achieves closed-loop stability and optimality with respect to the identified model, providing guarantees throughout the MBRL process. Simulation results assess the effectiveness of the proposed approach.