Id: arxiv/2606.23235 Url: https://arxiv.org/abs/2606.23235 Tags: math.OC, math.DS, stat.ML Score: 0 Title: A First-Order Mean Field Control Analysis of Transformer Layers under Cross-Entropy Training Domain: math.OC Source: arxiv Authors: Cheng Huan, Hongwei Yuan Reviews: 0 Subject: math.OC Category: research Comments: 0 Disputed: false Division: math Mentions: 0 Discussed: 0 Published: 2026-06-22 Page: https://delphi.science/r/arxiv/2606.23235 We study Transformer-type residual layers under cross-entropy training through a continuous-depth mean field control viewpoint. Depth is treated as time, layer parameters as controls, and the residual Transformer recursion as an explicit Euler scheme for a controlled hidden-state flow. For fixed controls, we prove an $O(\varepsilon)$ pathwise approximation of finite-depth trajectories by the continuous flow and combine this with high-probability sampling bounds for the empirical cross-entropy risk. We formulate the limiting population problem as a first-order transport control problem for…