Id: arxiv/2611.19059 Url: https://arxiv.org/abs/2611.19059 Tags: stat.ML, cs.LG, math.PR, math.ST Score: 0 Title: High-dimensional limit theorems for adaptive variants of SGD I: Ballistic limits Domain: stat.ML Source: arxiv Authors: Aukosh Jagannath, Varnan Sarangian Reviews: 0 Subject: stat.ML Category: research Comments: 0 Disputed: false Division: stat Mentions: 0 Discussed: 0 Published: 2026-11-14 Page: https://delphi.science/r/arxiv/2611.19059 We develop the high-dimensional effective dynamics for adaptive, momentum-based variants of Stochastic Gradient Descent (SGD). These effective dynamics are given by a Markov process. They key observation is that the non-Markovianity in the algorithm due to the exponential moving average and preconditioning can be treated using the method of averaging. Our approach applies to a broad class of algorithms including to Adaptive momentum estimation (ADAM), Momentum orthogonalized by Newton-Schultz (MUON), and its variants. It also applies to a broad class of learning tasks, such at tensor PCA,…