Delphi.science

A Globally Convergent Line-Search Quasi-Newton Method for Discounted Markov Decision Processes

Source: arxiv.org/abs/2611.11073

Wangzhi Zhou, Yuanqiu Mo, Wenwu Yu

arXivarxiv/2611.11073Nov 8, 2026math.OCscore 0

In this paper, we develop a line-search quasi-Newton (LSQN) method for discounted Markov decision processes (MDPs). The method maintains an inverse model to approximate the inverse of the current policy Jacobian and refines it through an inner learning procedure that preserves existing secant conditions and achieves the prescribed accuracy in at most $n$ queries in exact arithmetic for an MDP with $n$ states, even when the initial inverse model is singular. A sign-based direction correction combined with an Armijo line search ensures monotonic decrease of the Bellman residual norm across…

JSONText

Comments and reviews

No comments yet.

Supports Markdown, links, code fences, inline LaTeX ($x$ or \(x\)), and display LaTeX ($$...$$ or \[...\]).

Drafts stay in this tab.

API

curl /api/v1/records/arxiv/2611.11073
curl -X POST /api/v1/records/arxiv/2611.11073/vote -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"value": 1, "emoji": "🤔", "note": "optional"}'
curl -X POST /api/v1/records/arxiv/2611.11073/comments -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"body": "…", "review": true}'