Jury Methodology: A Pre-Registered Results Pack

Abstract

Jury Methodology: A Pre-Registered Results Pack. How should an AI jury be evaluated without publishing the instruction library that makes the jury useful? This report's answer: preregister the comparison, keep the judging recipe sealed, publish result-level evidence, and preserve the failures alongside the successes. The pack records a pre-run specification followed by a post-run result on 30 July 2026 — six cases, two judging lenses, two arms, twenty-four calls, with intended measurements and a conditional decision rule fixed before outcomes were inspected. The central limitation is binding, not ceremonial: only six cases, both arms in the same model family. Every comparative interpretation is a hypothesis or tentative signal; the sample is insufficient for defensible agreement estimates, and the report makes no conclusive ranking. The planned false-positive endpoint failed because its reference had decayed. The most defensible outcome is a prospectively specified, auditable hypothesis for a hybrid jury — not a settled model ranking. Companion to the jury paper The Jury That May Not Vote for Its Own Family (10.5281/zenodo.22087249) and the campaign paper (10.5281/zenodo.22086013). Bilingual: full text in English and Turkish. AI-transparency note: an AI system assisted with source synthesis and drafting under human-defined confidentiality and evidence constraints; the named authors remain responsible for interpretation, verification, and publication. Blind cross-family jury-reviewed (producing families excluded); the judging-instruction library is deliberately withheld (citable, not copyable).