When Can Orthodontic Over-Treatment Be Measured? A Pre-Registered Identification Assessment, a Falsified Claim, and a Cross-Sectional Analysis of Untreated Occlusal Need in NHANES III

Abstract

Background/Objectives: Orthodontic care is widely described as simultaneously over-supplied to those with little clinical need and under-supplied to those with much. Testing the first half of that claim requires a data source in which need is recorded independently of entitlement. We asked whether any such source exists, and, where need can be measured independently, how much of it remains untreated and how that residual is distributed socially. Methods: We pre-registered four conditions under which population-level over-treatment is identified from a data source: need measured before treatment on the same individuals (I1); a sampling frame containing treated and untreated individuals (I2); need measurement not conditioned on treatment eligibility or reimbursement (I3); and treatment receipt observed irrespective of payer (I4). The pre-registered falsification rule was that if any source satisfied all four, the central claim would be reported as falsified. At reviewers’ request, we replaced the original four-source convenience assessment with a documented search: 112 requests logged across eight frames, 15,341 records identified, and 14,139 records screened. Residual untreated occlusal need and its social gradient were estimated in a cross-sectional analysis of the Third National Health and Nutrition Examination Survey (NHANES III, 1988–1994) using a design-informed Bayesian hierarchical logistic model, cross-checked against Taylor-linearised design-based estimation. Results: The falsification rule fired. Of 26 sources assessed against the four conditions, 25 fail—and their failures fall into the two families the framework distinguishes: administrative sources on I3 and population surveys on I1—but one satisfies all four. The US National Health Examination Survey Cycles II (1963–65) and III (1966–70) do so because the two cycles are individually linked, giving a pre-adolescent occlusal record on 2177 individuals in the adolescent sample; excluding those already in appliance therapy at Cycle II leaves 2075 individuals. The source is open. Estimand E2 is therefore falsified, and the universal claim is withdrawn; the mechanism it describes is not. In NHANES III, weighted need prevalence among 4593 children aged 8–17 ranged from 23.4% to 44.5% across four composites—the range is the finding, and the point estimate is secondary to it—with 33.3% (31.4% to 35.2%) under the registered composite. Of children with need present, 87.8% had never been treated. Standardised risk of untreated need—a population-standardised posterior quantity, not an observed proportion—fell from 37.6% at the most disadvantaged income rank to 21.7% at the most advantaged: a risk difference of 15.9 pp (11.1 pp to 20.6 pp). The gradient survives a differential report-error correction up to 2.0× and agrees with design-based estimation to within 0.046 pp. Conclusions: Over-treatment is not identified by the need-gated administrative records from which it is usually estimated; the reason is structural: eligibility rules that respond to perceived over-treatment destroy the evidence of what they detect. It is identified by a linked open survey fielded sixty years ago, which is why the stronger claim that no such source exists does not survive. Untreated occlusal need, by contrast, is measurable, substantial, and socially graded.