The Second Loop: Durable Correction of Memorized Errors in Frozen Language Models

Abstract

A frozen language model that has memorized a wrong fact repeats it even when the correct fact is placed in its context. We study whether such an error can be corrected durably — surviving a full process reload — without touching a single weight, and where the criterion of truth for such a correction must live. We introduce the scar-survival rate: the fraction of trap facts on which an external correction mechanism keeps beating the model's memorized instinct across N full reloads, scored by an independent semantic judge over the generated answer. On a frozen Qwen2.5-3B-Instruct subject, a per-fact notebook with contrastive decoding turns 0/14 correct answers into 14/14 and holds them across ten reloads — 100% under a deliberately deterministic setup (frozen weights, static notebook, greedy decoding) — and under stochastic-decoding stress (700 samples at temperature 0.7), with 92.9% under up to +500 entries of notebook ballast. The mechanism has one sharp failure mode: it is blind to truth. A single plausible lie placed next to the truth halves survival to 50%. A gatekeeper built from a same-family model cannot close the gap — it inherits the subject's blind spots and caps at 64.3%, exactly its own accuracy — whereas a gatekeeper grounded in live Wikipedia plus targeted retrieval engineering restores 92.9% (13/14), and 100% once one genuinely ambiguous trap is disambiguated. The metric ports across three models and two fact domains (100% in all six cells, 540 answers) and discriminates a real correction from cosmetic and placebo variants (100% / 0% / 14.3%). A separate probe asks the opposite question — how rarely external truth can arrive before a self-calibrating frozen forecaster drifts — over 1,646 resolved ForecastBench questions and eleven reveal schedules: any finite contact holds; only zero contact fails. We read the whole arc through Tarski's undefinability of truth as an organizing analogy: neither the subject nor its same-family copy can supply a working criterion of truth; a properly integrated external source can.