Reme Degen

Two Blind Judges, One Arbiter: A Reliability Audit of LLM Labeling Decisions in a Production Pipeline

A methods report auditing a production pipeline that labels short financial news with three-class sentiment using two blind LLM judges from different model families (Claude Opus 5, GPT-5.6) and a third model (Claude Fable 5.1) as arbiter. It analyzes 34,998 judged rows, a 1,250-i...