Uniqueness in Deep Neural Networks: Chaotic Dynamics and Irreproducible Trajectories
Ghiassian, Haamed · Zenodo (CERN European Organization for Nuclear Research) · 2025
Deep Neural Networks (DNNs) exhibit a dual behavior: on one hand, they demonstrate robust convergence to low-error solutions in the output space; on the other, they display extreme sensitivity to initial conditions in the weight space. This paper argues that the training dynamics of DNNs constitute a chaotic system characterized by unique, non-repeating trajectories. By modeling the gradient descent process as a discrete dynamical system, we show that the solution manifold acts as a strange attractor—abounded region with fractal structure where dynamical trajectories converge macroscopically yet remain microscopically chaotic and divergent. Leveraging Lyapunov analysis and the dynamics of chaotic systems, we provide a theoretical framework proving that every trained network is a topologically unique realization, distinct from any other instance. This uniqueness renders both the trajectory and the final state of a neural network fundamentally non-reproducible.