Id: arxiv/2611.10691 Url: https://arxiv.org/abs/2611.10691 Tags: cs.LG, math-ph, math.PR, nlin.AO, stat.ML Score: 0 Title: A Geometric Theory for Nonequilibrium Transformer Dynamics Domain: cs.LG Source: arxiv Authors: Aythami Bethencourt de León, Darryl D. Holm Reviews: 0 Subject: cs.LG Category: research Comments: 0 Disputed: false Division: cs Mentions: 0 Discussed: 0 Published: 2026-11-07 Page: https://delphi.science/r/arxiv/2611.10691 We develop a variational and geometric theory based on statistical physics for the joint evolution of Transformer attention and token representations at fixed model parameters. To this end, we introduce a free energy for the attention and model the evolution of the attention weights by its Fisher--Rao gradient flow. At fixed token representations, we prove that the softmax matrix is the unique free-energy minimiser and that every positive attention trajectory converges to it. For prescribed time-dependent representations, we show that the attention dynamics becomes linear after a change to…