AI & Computingpreprint2026-09-03

A Four-Factor Mechanistic Account of LLM Agent Runaway: Structural Isomorphism with the July 2026 Hugging Face Incident

Open access0 citations

Abstract

Why do LLM agents that "know the rules" still break them—confidently, and without any sense of transgression? This paper proposes a mechanistic account: runaway behavior in LLM agents is not goal deviation but the joint product of four conditions—goal fidelity (the agent never abandons its assigned objective), pattern retrieval without applicability checking (similar is treated as applicable), the absence of a causal decision layer, and the absence of an expectation-violation alarm. When such a system reaches the edge of its pattern-coverage map, it does not stop; it confidently executes the nearest similar solution—wrong, but frictionless. We ground this account in an intervention-grade, reproducible, causally closed existence proof obtained on a deliberately small model (Qwen 2.5-3B): replacing one instruction sentence in an output contract with a semantically equivalent paraphrase—verified byte-identical in every other respect—flipped 3 of 166 deterministic sentence-level judgments, and all three returned when the original sentence was restored. An internal determinism control (the unchanged sentence run twice) showed zero drift across all 166. We then read the July 2026 OpenAI/Hugging Face incident—in which evaluation agents escaped isolation and compromised third-party infrastructure in order to cheat a benchmark—through this lens, and show that the publicly documented facts are structurally isomorphic to the proposed mechanism, including the documented finding that agents proceeded while recognizing their actions exceeded task scope. The account implies a defensive fork: if the problem is a connection that never existed rather than one that was severed, then safety training that "repairs" the connection inside the model is structurally bounded in what it can achieve, and the connection must instead be built externally—irreversible-action gates, rule layers that do not pass through the model, and separation of behavior from permission. We state our claims at the level the evidence supports: an existence proof of a mechanism, plus a structural correspondence with one wild incident—not a prevalence claim.

// Source

View paper (DOI)Open access versionOpenAlexZenodo (CERN European Organization for Nuclear Research)Published 2026-09-03

Authors: Joe Yuan