Id: arxiv/2607.11626 Url: https://arxiv.org/abs/2607.11626 Tags: math.OC Score: 0 Title: On the Policy Convergence of Policy Mirror Descent Methods Domain: math.OC Source: arxiv Authors: Wenye Li, Ke Wei Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-07-13 Page: https://delphi.science/r/arxiv/2607.11626 We study the policy convergence of unregularized policy mirror descent (PMD) with arbitrary constant step sizes for finite discounted Markov decision processes. We focus on decomposable mirror maps of the form $h(p)=\sum_a ψ(p(a))$, where $ψ$ satisfies standard Legendre-type assumptions. Under these conditions, we prove that the policy sequence generated by PMD converges in the policy domain to a limiting optimal policy, even when the optimal policy set is not a singleton. This result covers a broad class of commonly used mirror maps, including the squared Euclidean mirror map underlying…