Breaking the Selection Link: A Pre-Registered Validation Audit of the Factor Zoo

Abstract

Version 2 (9 August 2026). This version corrects the strength of one interpretive claim and adds a post-hoc addendum; no committed or graded quantity changes. Version 1 stated that the sign channel bounds the null share of the published record well below 80%. The formal bound, constructed in the addendum following correspondence on version 1, is materially weaker than the point estimates and rests on the null transition law; the claim is restated as the point estimate the note's Section 5 always labeled it. The addendum reports the drift-priced bound on both panels under the panels' own resampled dependence, together with the persistence-family and joint inversion-and-certification diagnostics. The new file bound_archive_v2.zip carries the bound engine, both critical-drift matrices, and the diagnostic scripts (code and derived statistics only; no return series). Version 1 and its archive remain unchanged on the version 1 record. Abstract. López de Prado and Fabozzi (2026) prove that the false discovery rate of the factor literature cannot be identified from reported in-sample statistics under latent search and selection, and name independent validation as one of two remedies. This note reports a pre-registered validation audit that implements that remedy. Before touching any factor return series, we froze a per-factor validation design (publication-year and sample-end splits, Newey-West t statistics, a BH(0.10) certification gate, and a four-bucket ledger) and committed quantitative predictions under three calibrated worlds spanning the published debate: 6.3% false with 88% retention (Chen-Zimmermann), all true with 42% retention (McLean-Pontiff), and 80% false (the López de Prado-Fabozzi search-adjusted calibration). On the 212 Chen-Zimmermann predictors, 64 of 210 tested certify post-publication; on 142 JKP-USA capped-value-weight factors, 3 certify. Every committed certified-count interval missed, in both datasets. The misses are asymmetric: certified counts fall far below the optimistic calibrations, while sign inversions fall far below the 80%-false calibration, implying a null share near 39% (CZ) and 22% (JKP) as point estimates. Under the maintained calibrations, the validation data reject both poles of the debate: the factor zoo does not look mostly false, and its true members retain far less than published statistics imply. A version 2 addendum constructs the formal bound behind the sign-channel estimate and prices its dependence on the null transition law; the committed design and every graded quantity are unchanged from version 1.