Measure the divergence, not the agreement: A manifesto for the Real User Cost Framework

Abstract

Contemporary user experience measurement collapses attitudinal and behavioural evidence into composite scores. This paper argues that the collapse destroys the most diagnostically valuable signal available: the direction and magnitude of divergence between what users do and what users report. Drawing on eight decades of attitude-behaviour research, on evidence-grading practice from clinical medicine, and on distribution reporting from performance engineering, it proposes a framework in which divergence is the primary diagnostic axis, every measurement carries an evidence-quality weight, that weight decays on software release and not on the calendar, and results are reported as distributions instead of means. The framework is falsifiable, and section 9 states the conditions under which it should be rejected. Section 12, added in version 2.2, extends the same rules to design itself: identities, interfaces and reports.