WEAVE: Forward-Written Constant-Size Matrix Memory with Runtime Multi-Hop Composition

Abstract

The long tail of world knowledge is what makes language-model pretraining expensive. Grammar and reasoning circuits are finite, but facts are not — and composing separately-learned facts demands data proportional to the transitive closure (O(N^k) for N atoms), because transformers do not compose what they were not trained to compose (the two-hop and reversal curses). WEAVE breaks this coupling by separating knowledge from skill. A constant-size D×D matrix memory S acquires facts through forward-only outer products — one pass, zero backpropagation — and composes them at inference time via S^k with a per-hop codebook snap. Weights learn only skill: grammar, extraction, and the walk. The novelty is (1) runtime S^k composition of forward-written facts, and (2) four capabilities — gradient-free write, runtime compose, targeted unlearn, and abstain — unified in one constant forward-written substrate. In a controlled head-to-head on identical single-hop facts (N<=1000), WEAVE achieves 100% zero-shot two- and three-hop composition where a transformer memorizes training hops perfectly but scores 0% on held-out two-hop queries (the two-hop curse). Built-in negative controls — shuffled linkage and direct cross-hop reads both scoring 0% — confirm genuine chain traversal. Reasoning depth decouples from network depth: S^k runs in zero layers while the language-model trunk knee is L4. One-shot recall reaches 37.9% versus 0.7% for gradient training. 2000 sequential read-erase-write edits accumulate no damage, where ROME/MEMIT collapses after dozens. An end-to-end conversational demonstration (49M-parameter trunk on Volta-16GB hardware, store logit injection) raises seeded-fact probability from <=0.08 to >=0.96 while unseeded entities remain at <=0.0002 — abstention, not hallucination. This is a preprint. Independent research. Experiment code available on request.