Reading More, Finding Less: A Pre-Registered Anatomy of Mis-Routing in Curated Agent Retrieval

Abstract

This paper measures how well a curated index routes an AI agent to the right information. Its instrument differs from standard retrieval evaluation: instead of scoring a retriever's returned set (recall@k), it records, per question, whether the agent's own read actions reached the correct document. Agent systems commonly provide curated summaries and index files, a pattern known as progressive disclosure, on the assumption that they help the agent reach the correct content at low cost. Recent studies compare curated access against search, but published designs measure their curated arms by aggregate outcome only, without per-read telemetry, so a curated arm's failure cannot be attributed to the index's routing; the closest preregistered neighbor names direct read telemetry as a target for replication. This study supplies that instrument: in the curated experimental arm, an authored one-line-digest index is the agent's only way to locate documents, and every question records whether the policy reached the correct document. The measured result is a decomposition of the failure. The index routed the agent to the correct document on 52.0% of 102 questions; associative search (grep) over the same corpus located it on 80.4%. When the curated agent answered incorrectly without the correct document in context, it had read a wrong file in 43 of 47 cases and read nothing in 4. When the index did route correctly, accuracy was 86.8%. A difficulty-matched control on the oracle arm shows the questions the index failed to route are equally answerable (Fisher p = 0.80). The deficit of the curated policy is localization, not comprehension. A pre-registered public replication over 141 releasable documents and 120 frozen questions reproduced the direction under stricter criteria: search led by 12.5 percentage points on accuracy (63.3% versus 50.8%), wrong-stop rates were 34.2% versus 18.3% under a symmetric rule, and localization was 75.0% versus 62.5%. The replication's formal verdict is "revised" because one hardened prediction about token headroom failed; the paper reports that failure at the same prominence as the passing predictions. In the same measurement program, files promoted into a durable memory directory were later read in 2 of 157 eligible cases, and the corresponding conjecture is refuted; the paper states the evidential asymmetry between the two results rather than averaging them. All predictions were frozen before data collection, analyzers were committed before their first output, and adjudication is mechanical against the frozen criteria. The deposit contains the paper (PDF, HTML, and canonical markdown), both figures, the reference list, the full change history between drafts, and the complete replication study: the corpus snapshot, the frozen questions with gold answers, the authored index, the harness, the adjudicator, and all run records. Every replication statistic re-derives from these artifacts byte for byte. This version supersedes its immediate predecessor (DOI 10.5281/zenodo.21947917) on a related-work attribution corrected in CHANGES-v3.1.md, after an external audit verified against sources. Version 1 of this record ("Agent Memory Allocation: Tiers and Policies of Effective Agent Memory", DOI 10.5281/zenodo.21938413) remains the version of record for the tier framework and the additional ecosystem measurements cited by this version.