Adam Allcock

ClerkBench: Measuring the Consistency Horizon of Language Models on Long Deterministic Workloads

Frontier language models can be highly accurate on a task yet fail to perform it reliably enough to run unattended. On long chains of simple, fully specified clerical computation, such as billing, payroll and ledger replay, the binding constraint is not capability but consistency...

ClerkBench: Measuring the Consistency Horizon of Language Models on Long Deterministic Workloads

Frontier language models can be highly accurate on a task yet fail to perform it reliably enough to run unattended. On long chains of simple, fully specified clerical computation, such as billing, payroll and ledger replay, the binding constraint is not capability but consistency...

ObviousBench: The Cost of Not Making Obvious Mistakes

Language models deployed in public-facing products still make mistakes any careful person would catch instantly: miscounting the letters in a word, botching a trivial spelling transform, ignoring a plainly stated constraint. These visible failures erode user trust faster than fai...

ObviousBench: The Cost of Not Making Obvious Mistakes

Language models deployed in public-facing products still make mistakes any careful person would catch instantly: miscounting the letters in a word, botching a trivial spelling transform, ignoring a plainly stated constraint. These visible failures erode user trust faster than fai...