Advances in Implicit Neural Models for Efficient Computation
橋本 尚典 · Institutional Repositories DataBase (IRDB)
This dissertation addresses fundamental challenges regarding computational efficiency, data efficiency, and representational capacity in "Implicit Neural Models (INMs)."Unlike explicit feedforward models, INMs define input-output relationships through internal state equilibria or dynamical processes, such as energy minimization and differential equations.While this formulation offers theoretical advantages, it often suffers from high computational costs and training difficulties when treated as a black-box function approximator.The central thesis of this research is that introducing appropriate structures or design principles tailored to the characteristics of each model is essential for transforming INMs into practical tools.Specifically, this dissertation constructs mathematical models and proposes novel algorithms focusing on Boltzmann Machines, Neural Ordinary Differential Equations (Neural ODEs), and Hopfield Networks.The dissertation is organized as follows.Chapter 1 establishes that the methods discussed belong to the unified framework of INMs and clarifies their common mathematical characteristics and the challenges impeding their real-world application.Chapter 2 proposes an unsupervised anomaly detection method for time-series data using Boltzmann Machines.To address the computational intractability of the partition function in conventional energybased models, the proposed approach introduces a change score derived from the difference in free energy.This formulation eliminates the need for expensive sampling, enabling linear-time execution while maintaining high sensitivity to changes in system dynamics.Chapter 3 addresses the data inefficiency of Neural ODEs in learning dynamical systems.By explicitly incorporating physically motivated constraints-specifically, the second-order kinematic relationship between coordinates and velocities, and the linearity of control inputs-into the i