Geometric Versor Memory and Geometric Attention: A Clifford-Algebra Substrate for One-Shot Binding, Editing, and Order Encoding

Abstract

Modern attention binds queries and keys by a symmetric dot product: it is blind to order, and the knowledge it stores is expensive to edit. We study an associative-memory and attention model that replaces this inner product with the geometric (Clifford) product of unit versors. Binding is a single geometric product, recall is multiplication by the versor inverse, and—because that inverse is exact—a stored fact is deleted exactly, with no retraining. We measure the model on small algebras Cl(n,0), n ≤ 8, under a reproducible synthetic protocol, and report both what it delivers and what it does not. A single field has low, sub-linearly scaling capacity dominated by superposition cross-talk (E1), which a sparse addressable arrangement restores to linear (E2); the pure model generalizes by robustness and exact analogy but needs a softmax readout to interpolate (E3–E4); and against classical one-shot and trained baselines it loses in raw capacity and noise robustness, its measured edge being storage density and composable algebraic structure (E0). Two trained-model results follow. Versor position binding is provably orthogonal, generalizes rotary position embeddings (RoPE), and matches them once the frequency spectrum is designed (E6). And, as our flagship result, under sequential knowledge editing a frozen-model versor memory surpasses a faithful rank-one (ROME) baseline on all three canonical metrics at 60 accumulated edits—efficacy 1.00 vs. 0.82, paraphrase 0.70 vs. 0.43, locality 0.98 vs. 0.92—with O(1) edit cost and exact deletion (E7). We conclude that the model is best understood not as a one-shot large language model but as an edit-friendly, order-aware memory–binding substrate for attention, in which the attention inner product is the geometric product. All results are small-model and synthetic; scaling to pretrained language models and standard benchmarks is left to future work.