Id: arxiv/2611.19850 Url: https://arxiv.org/abs/2611.19850 Tags: stat.ML, cs.LG, math.ST Score: 0 Title: Sharp Gaussian Approximation for Q-Learning Domain: stat.ML Source: arxiv Authors: Xingtu Liu Reviews: 0 Subject: stat.ML Category: research Comments: 0 Disputed: false Division: stat Mentions: 0 Discussed: 0 Published: 2026-11-15 Page: https://delphi.science/r/arxiv/2611.19850 Gaussian approximation provides a foundation for online statistical inference in reinforcement learning. We establish sharp non-asymptotic Gaussian approximation rates in Wasserstein distance for asynchronous Q-learning with Polyak-Ruppert averaging. We prove an upper bound of $\widetilde{O}(n^{-1/4})$, where $n$ is the number of iterations, together with a matching lower bound up to logarithmic factors. Q-learning is more challenging to analyze than linear stochastic approximation due to the nonlinearity of the Bellman operator. Asynchronous updates introduce further challenges due to…