Spectral regularization dynamics: a continuous-time framework for non-convex optimization
International Journal of Machine Learning and Cybernetics, vol.17, no.7, 2026 (SCI-Expanded, Scopus)
- Publication Type: Article / Article
- Volume: 17 Issue: 7
- Publication Date: 2026
- Doi Number: 10.1007/s13042-026-03164-8
- Journal Name: International Journal of Machine Learning and Cybernetics
- Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Compendex, INSPEC, Materials Science & Engineering Collection (ProQuest), Technology Collection (ProQuest)
- Keywords: Continuous-time dynamics, Dynamical systems, Non-convex optimization, Saddle-point avoidance, Second-order methods
- Eskisehir Osmangazi University Affiliated: Yes
Abstract
We propose Spectral Regularization Dynamics (SRD), a continuous-time system for unconstrained non-convex optimization. Unlike discrete iterations that solve regularized subproblems, SRD employs an autonomous feedback control mechanism coupled to the Hessian’s minimum eigenvalue. This mechanism guarantees a descent direction by driving the regularization parameter strictly positive in regions of negative curvature. We prove global convergence to the set of critical points via LaSalle’s invariance principle. Furthermore, using the stable manifold theorem, we establish that the system dynamics almost surely avoid strict saddle points. In local neighborhoods of strict minima, the regularization vanishes asymptotically, recovering the convergence rate of the continuous Newton flow. Numerical benchmarks illustrate the saddle-avoidance mechanism and local convergence properties.