PhysQK-AD: Convection-Diffusion-Driven Token Coordinate Evolution — From Pure Diffusion to a Learnable Convection-Diffusion Mixture

Abstract

Modern language models are built almost entirely on Transformer self-attention. PhysQK (published separately, DOI 10.5281/zenodo.21870472) proposed an orthogonal path: retain the standard Transformer stack (windowed self-attention + FFN), and augment it with pure-diffusion dynamics acting on token coordinates r: r_new = r + W_update(attn·r − r). This paper proposes its extension PhysQK-AD, generalizing pure diffusion to a learnable convection-diffusion mixture, r_new = r + η[(1−α)·W_update(attn·r − r) + α·(r[t]−r[t−1])] where α = σ(w_α) ∈ [0,1] is a per-layer learnable convection/diffusion mixing ratio, η = exp(w_η) is a learnable step size, and the convection term r[t]−r[t−1] is the intra-sequence spatial gradient ∂r/∂x. Through gradient voting, SGD decides per layer whether each layer is "convective" or "diffusive." In controlled experiments (d=256, 4 layers, 20 epochs, seq=128), PhysQK-AD is the only condition among the three that beats multi-head attention (MHA) at d=256 (ΔEval = −0.065), whereas pure-diffusion PhysQK at the same scale loses to MHA (+0.007) — the convection term is what pulls coordinate evolution back from "degradation" to "overtaking." The most robust finding is single-layer convection specialization: SGD consistently concentrates convection in L1 (α≈0.44 at 256d/20ep) and pushes L2/L3 toward closure (α<0.003 by 20ep), a single-layer-dominant pattern that holds across dimensions (128d/256d), training lengths (5/20 epochs), and seeds (3). Together with a Péclet-number diagnostic (L1's eff_Pe = 1.142 crossing 1.0 for the first time, synchronized with the performance gain) and an α-allocation ablation (any simpler allocation beats joint per-layer learning by 0.43–0.51 nats), this paper offers feasibility evidence for the "convection-diffusion equations drive token-coordinate evolution" path, plus one cheap design rule. This is a condensed preprint; an extended version will follow.