Tests found that a model’s sense of how believable a text sounds could stay high as factual accuracy declined.
Researchers examined 274 source texts, including recent and older news articles and historical articles. They created controlled versions that differed in factual accuracy and presentation, then tested how the language model judged them.
The model’s overall reliability score stayed relatively high as accuracy fell, and completely fabricated texts received mean scores above 3 on a 1–5 scale. When researchers separately tested factual correspondence and the credibility conveyed by the writing, factual judgments were more accurate, but performance varied substantially with how familiar the information was.
When credibility outlasted facts
In the first phase, a single language model’s reliability score remained relatively high even as the factual accuracy of texts decreased. Completely fabricated texts still received mean scores above 3 on a 1–5 scale. In some cases, the model gave a high numerical reliability score while its written explanation identified substantial factual problems.
In the second phase, researchers separately assessed whether texts matched external facts and how credible or justified they appeared. The factual assessment distinguished authentic from fabricated texts better than the credibility assessment, with an overall AUC of 0.801, sensitivity of 0.766 and specificity of 0.810. Performance varied with information familiarity: it was near chance for newer CNN articles and nearly perfect for older and historical material.
Why the distinction matters
The findings suggest that an AI language model can preserve an appearance of credibility as factual grounding is weakened. Asking the model to focus directly on whether claims correspond to external facts improved discrimination, but did not fully separate factual knowledge from familiarity, plausibility and other cues in the writing.
This distinction matters when language models are used to assess information in settings such as education or health care. A confident-sounding judgment may not reliably show that the underlying claims are true.
Evidence and caveats
The study used 274 source texts and controlled authentic and fabricated variants, producing 548 observations for the separate factual and credibility assessments. It reports results from the tested language model and dataset, rather than establishing how all language models would perform.
Performance also changed sharply with information familiarity, and the abstract does not specify the model, the exact text-generation procedure or how results would transfer to real-world misinformation. The findings therefore support a distinction between factual correspondence and perceived credibility, but do not define how the results generalize beyond these materials and tests.
// Source
Journal of High School Science · 2026 · DOI: 10.64336/001c.169559
Authors: Haoning Luo
Institutions: Shanghai Jinyuan Senior High School
