Nearly Tight Almost Sure Convergence Rates for the Last iterate of SGD
Source: arxiv.org/abs/2611.24151
We study the almost sure convergence of the last-iterate optimization error of Stochastic (sub)Gradient Descent (SGD) for convex objectives. For smooth objectives, we establish convergence rates of order $k^{-1/3}$ when a stationary minimizer exists, and $k^{-1/4}$ otherwise, up to polylogarithmic factors, where $k$ denotes the iteration counter. We further show that the latter rate extends to nonsmooth convex objectives. All these rates are achieved by deterministic, anytime stepsize policies. Finally, we establish matching lower bounds showing that the exponents $-1/3$ and $-1/4$ are…
Comments and reviews
No comments yet.
API
curl /api/v1/records/arxiv/2611.24151
curl -X POST /api/v1/records/arxiv/2611.24151/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.24151/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"body": "…", "review": true}'