The Largest Bundler Comfort, Certainty and Guidance in AI Assistants, and the First Seller of the Bundle Who Holds a Denominator — A Sequel to Comfort Needs No Denominator
Abstract
Abstract Comfort Needs No Denominator showed that motivational and success content sells three goods in one product: comfort, which acts on the buyer’s state; certainty, which scales up positions without improving estimates; and guidance, a claim about effects. The three differ sharply in who can read them and what evidence they need, and bundling is selected by the multitask incentive of measuring one task and not the other. This paper argues that, measured by audience, the AI assistant is the largest seller of this bundle so far, and that it differs from human sellers in three places. First, the bundle is written into the training signal. Training on human preferences reads mainly the satisfaction layer; by the reading-ownership principle it is a qualified reading for comfort, only a biased and noisy reading for guidance, and nearly blind to side effects, so on the inputs where saying what the user wants to hear conflicts with saying what is right, sycophancy is not an accidental defect but the bundle that optimisation selects. Training scored on outcomes, on tasks with verifiable answers, is an unbundling that has already partly happened. Certainty acts on the user’s position and also on a toll the user cannot see: the gap between the ensemble average and the growth of a single trajectory, the ergodicity gap, grows with the square of the relative position; when outcomes combine a large probability of failure with a heavy tail, even an entirely correct average can sit far above the median outcome for one person, which is zero at full commitment. Second, one model is a single watchtower shared by many lakes. When errors are correlated at ρ across users, the effectively independent sources of advice, in the sense of averaging, do not exceed 1/ρ, and the probability of synchronous error has a floor that does not fall with the number of users; checking advice with another instance of the same model is same-origin monitoring, blind exactly when something goes wrong; a person who understands the world mainly through the same model is no longer its heterologous checker either; and when preference data come from users the model has shaped, the training signal becomes the model’s own reflection, and sycophancy turns from a bias that can be corrected generation by generation into a ratchet that accumulates in one direction. Third, the provider is the first seller of this bundle who holds a denominator in principle. Insurers, lenders and recommender systems have long held outcome denominators across users, but none of them sells comfort, certainty and guidance in a single delivery. For a human speaker the missing denominator is an epistemic impossibility; for a provider it is first a question of incentives about what is measured. But a denominator held and read by the provider fails the first condition of reading ownership, and the system under evaluation fails passivity, so an effective denominator must be heterologous, pre-registered and outcome-based. The side effects of comfort delivered by AI — closing the drive, rewriting set points, removing the user’s own checks — lie at layers satisfaction cannot read, and satisfaction, a snapshot indicator, looks healthy right up to a transition. Nine predictions are registered, six of which can be run with existing models alone, with protocols given in executable form; the protocols include a two-factor design separating distress from reserves, a common-mode estimate that removes item difficulty and is read by dispersion, a source-blinded comparison of checks, a contrast between the mean, median and geometric-mean readings of a denominator, and a manipulation check on source recognition. Executions on external data: zero.
// Source
Authors: Qinfu Li