Joshua Webber

CRLN-PRE-002 — Pre-registration: can a model report low confidence if the scale actually invites it?

A pre-registration, published before any response in the study was collected. An exploratory analysis of 9,393 stored responses from five language models on a clinical research competency instrument found that confidence levels 1 and 2 on a five point scale were used zero times....

CRLN-AIE-001 v0.2 — What an instrument with a measured ceiling can and cannot tell you about frontier models

Version 0.2 of the CRLN AI competency evaluation protocol, correcting the central result of v0.1. Correction. v0.1 compared an unweighted model mean against a response-weighted human mean and concluded that a frontier model and competent practitioners were indistinguishable. Thos...