Popularity Is Not Association: Auditing a Belief-Forming Rule Against 19,668 Respondents

Abstract

A system that forms beliefs about people from partial evidence was tested against a population whose correlation structure the author did not choose. Two defects were found, and both were load-bearing. The rule that mined regularities recovered the dataset's known five-factor structure at 0.185, against a chance rate of 0.184. It was measuring how common a claim was, not whether two claims were associated. Separately, the confidence attached to a newly formed belief was a linear score rather than a probability, and scored substantially worse than a procedure that ignores all evidence and predicts the population base rate. Both were repaired and re-measured. Recovery of the latent structure rose to 0.875, using 94 per cent fewer regularities that covered a larger number of claims than the unrevised rule. Assigned confidence reached parity with the base rate and did not exceed it; the paper is explicit that this constitutes the removal of a negative rather than a demonstrated positive. The computational behaviour of the revised system is reported, together with a correction to a result this programme had published a week earlier: the pooled bank had been said to fail in a saturated population, and the measurement was real while the explanation was wrong. A dated note added 3 September 2026 records that positive skill has since been established, and identifies the two further defects responsible. The original text is left standing rather than rewritten. Data: Open Psychometrics Big Five item responses, retrieved at run time and not redistributed. All harnesses are runnable in the reference implementation.