PhysQK-AD: Convection-Diffusion-Driven Token Coordinate Evolution — From Pure Diffusion to a Learnable Convection-Diffusion Mixture

Abstract

We report PhysQK-AD, a sequence model in which token coordinates evolve under convection–diffusion dynamics. In a small controlled setting (d=256, 4 layers, seq=128, 20 epochs), pure-diffusion PhysQK is close to a same-size multi-head attention baseline in validation loss (Δ ≈ +0.007), while adding convection–diffusion (PhysQK-AD) yields a lower single-seed validation loss (0.065 lower than MHA, but ~7% more parameters); training wall-clock is longer (pure diffusion ≈1.4×, AD ≈1.6× of MHA). These numbers, together with their respective qualifications, are listed in Section 3.2. This exploration nevertheless left two observations we think are worth writing down. First, a non-attention gate that we originally added as a secondary component — the per-head, per-channel channel-selection gate (CSG) — shows a stable load-bearing effect in ablation: removing it raises eval by 0.0516 ± 0.0063 (n=3), whereas removing the coordinate-evolution term alone is only 0.0204. Second, a series of alternatives to the coordinate-dynamics mechanism (convection–diffusion mixing, second-order inertia, collision box, diagonal SSM mixer) gave measured results that are mixed at this scale, which we list as-is in a quantitative table (Section 4). Because of a partial course correction, we tentatively position this paper as an exploration report and keep the "third way" question open.