Reading creativity from the inside: a competing-routes view of the diversity–coherence trade-off in language models

Abstract

Raising the temperature does not make a language model more creative, and this paper shows why at the level of the token race: temperature collapses value and coherence without ever raising novelty, while prompts that change which continuations can win raise judged novelty monotonically until value gives way — so usefulness (novelty × value) peaks at an interior breadth. This inverted-U replicates across four models (8B–235B) and a four-judge panel (concave in 15 of 16 model × judge cells). The instrument is a per-token count read from the logprobs of an ordinary forward pass: the number of candidate tokens above a probability threshold (“competing routes”, CR), examined at the choice point — the first generated tokens, where a continuation’s direction is set. Two further results. (1) The known early-position concentration of aligned models’ output distributions is replicated with this count-based instrument under a fixed-format matched-pair control in two of three families (Qwen2.5-7B, Mistral-7B-v0.1) and shown to be graded and post-training-recipe-dependent rather than universal: Llama-3.1-8B shows no genuine narrowing, and across families the size of the internal narrowing tracks the size of the behavioural-diversity collapse. (2) Choice-point CR gives a one-pass estimate of between-sample diversity (median within-model r = +0.34 against semantic spread, +0.54 against Distinct-2) — a positive relation where the nearest prior work reported an inconsistent null, with the discrepancy traced to range restriction rather than metric choice. The paper closes with a substrate-relative account of when a creativity intervention helps: to the degree the substrate does not already supply the operation the intervention performs. All generation, scoring, and analysis scripts are included, with per-response data (per-token CR, raw top-token logprobs, judge scores) as JSONL, in the accompanying archive. Target venue: Transactions on Machine Learning Research (TMLR). v2 (August 2026) — correction. The figures reported in the mechanism section are reconciled with the instrument that produced each: headline competing-route figures are the full-distribution values, while the entropy and coverage-mass figures are labelled inline as the truncated top-20 recomputation, and the umbrella sentence that conflated the two is corrected. A contrast previously described as directional is reported with its effect size and p-value as not significant at the stated level; the monotone attenuation across thresholds is stated explicitly with its numbers and read as a breadth change rather than a threshold artifact. The archived data-and-code package is updated. Earlier versions remain in the version history.