A Replayed Agent Memory Summary Keeps More Sessions on a Temporary Fallback Than the Run Log Does

Abstract

Coding agents rewrite past runs into a memory summary that later sessions act on. This paper shows, in an adapted replay of the memory pipeline of OpenAI's open-source Codex agent at a pinned commit, that the default (v1) summary keeps more sessions on a temporary fallback than a run log of the same runs does. Scripted runs of recurring tasks showed nine successes of a standard route, then two runs on a fallback route at the user's request for a check of it dated to those two days; fresh sessions of eight models, which saw only the summary or the log, chose the next run's route. On the readers' mean, 29.9% chose the fallback after the summary, against 3.3% after the log (a registered contrast, chosen as the headline after the results: +0.264, 95% interval +0.198 to +0.333 under a sign-flip model over sixteen fixed task domains, treating only the models' answers as random). Whether stating the check changes choices after the summary (the registered question) was not determined; two model coders rarely found the check or its scope in the summaries. A further registered study replaced one of three neutral lines under each summary with a line stating the check's purpose, the requested use's end or the standard route's record: against 35.9% with the neutral lines, the share choosing the fallback was 4.9% and 2.8% with the first two (both lower under the registered analysis) and 29.8% with the third (its difference undetermined), and below the log's with all three (0.6% against 2.9%; lower); the log beside the summary brought it to within 0.05 of the log's (2.7%, on the readers' mean). Each session saw one scripted history and one wording of the check and of each line, without memory tools; no choice is scored as right, and the gap's cause is not identified. Files. paper7-preprint-v1.pdf: the paper (60 pages), built with pdfLaTeX (TeX Live 2025) from the source archive. SUPPLEMENT.md: its supplement, which the paper cites as Sections and Tables S1, S2, .... paper7-latex-source-v1.tar.gz: the LaTeX source as built, with the ancillary files (the supplement, each study's registered analysis output and the comparisons with the independent re-implementations). SHA256SUMS: the SHA-256 of the other three files. Registration and data. Each study's protocol, code and materials were frozen under a hash manifest, deposited on Zenodo and timestamped in a Bitcoin block before its smoke and confirmatory calls: Study 1 doi:10.5281/zenodo.23291437; Study 2 doi:10.5281/zenodo.23335172 (its first registration, which ended at its smoke) and doi:10.5281/zenodo.23339829; Study 3 doi:10.5281/zenodo.23404871. Study 3's run records are public at doi:10.5281/zenodo.23411416; those of Studies 1 and 2 are archived by the author and are not public. The paper's Section 7 lists the departures from the registrations, among them that the headline contrast was chosen after the results and that the abstract reports Study 3's registered words through the cells' levels. AI assistance. The studies were chosen, designed, implemented, run and drafted by an AI coding assistant (Anthropic's Claude Opus 5.5, in Claude Code) acting for the author; OpenAI's Codex implemented Study 3's run chain on synthetic materials and recomputed Study 2's and Study 3's primary families independently. Every review of the designs, the results and the manuscript was made by a language model (Codex on GPT-6.1 Sol, separate read-only instances of Claude Opus 5.5, and critiques by another language model at the author's request). The studies' readers, writers and coders are language models. The author read the manuscript and is responsible for its content; no language model is an author. Suggested citation. Nakayashiki, K. (2026). A Replayed Agent Memory Summary Keeps More Sessions on a Temporary Fallback Than the Run Log Does (v1). Zenodo. https://doi.org/10.5281/zenodo.23427118