Id: arxiv/2607.21520 Url: https://arxiv.org/abs/2607.21520 Tags: math.OC, math.PR Score: 0 Title: Asymptotic Analysis of Empirical Dynamic Programming in Infinite-Horizon Stochastic Optimal Control Domain: math.OC Source: arxiv Authors: Xin Chen, Elif Sena Isik, Johannes Milz Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-07-23 Page: https://delphi.science/r/arxiv/2607.21520 We derive statistical limit theorems for sample-based approximations of infinite-horizon discounted stochastic optimal control problems in discrete time. Our first result is a functional central limit theorem for the sample-based value function under a uniqueness-type condition on population optimal policies. The limiting law is a mean-zero Gaussian process characterized by a linear fixed point equation that resembles a dynamic programming principle. We compare these asymptotics with those obtained from sample-based policy optimization and illustrate that their limiting variances can be…