The Probability Changes With the Model: What Density Estimation Actually Believes

Abstract

Probability models are often described as if they reveal how likely observations truly are. In practice, a density is conditional on a representation, model family, complexity, bandwidth, covariance rule, reference population, and measure. Even the numerical value of a continuous density changes with the coordinate system in which it is expressed. This paper evaluates 100 density models on the 307 development vehicles from the complete-case Auto MPG data. Four representations are combined with single-Gaussian models, Gaussian mixture models, and kernel density estimation. Five expanding validation windows ending in 1975–1979 select one model within each representation–family group, producing 12 primary models without consulting origin or the later temporal population. The 85 vehicles from 1980–1982 remain reserved for diagnostic temporal evaluation. Across the 12 primary models, mean cross-model density-score Spearman agreement is 0.559, with a minimum of 0.004. Mean Jaccard agreement between their 5% out-of-fold low-density tails is 0.327, with a minimum of 0.067. Models can therefore assign almost unrelated density rankings and identify substantially different observations as belonging to the same nominal low-density tail. Under 50 deterministic 1% feature-scale perturbations per primary model, mean score-rank stability is 0.990. Mean low-density-tail stability is lower at 0.904 and reaches a minimum of 0.768. A smooth and stable score does not necessarily produce an equally stable binary tail decision. When the frozen 5% development thresholds are applied to the 1980–1982 population, the mean low-density rate rises to 0.329 and ranges from 0.024 to 0.976. A nominal 5% development tail therefore does not imply a 5% tail in a later population. The largest post-fit normalized mutual information between Gaussian-mixture components and the withheld origin label is 0.416, but origin remains an external audit and does not select any model. The results distinguish predictive density, model-based typicality, cross-model ranking agreement, tail membership, perturbation stability, external alignment, and temporal transport. None establishes a uniquely correct density, intrinsic abnormality, natural classes, or a coordinate-free probability attached to each observation. The deposit includes the complete paper and a reproducibility package containing source code, data, machine-readable results for all 100 candidate models, five expanding-window validation records, 12 primary-model audits, 50 deterministic perturbation paths per primary model, two complete 12 × 12 agreement matrices, 48 Gaussian-mixture external audits, five generated figures, a SHA-256 integrity manifest, and an automated reproduction verifier.