FRAME: Faithful Multi-Agent Fact Checking via Hierarchical Gating and Bayesian-Inspired Evidence-Conflict Perception

Abstract

Fact checking is a key task in the field of natural language processing, and aims to judge the authenticity of a given claim by using external evidence. Existing multi-agent-based fact-checking systems achieve significant improvements on multiple benchmark datasets, but still have two core limitations: firstly, they lack a full-process faithfulness-verification mechanism to test the faithfulness of intermediate products at each stage, resulting in the amplification of hallucination errors cascading through the pipeline; secondly, when both supportive and refuting evidence exist, the system often relies on the implicit preferences of the model to make judgments. To address these issues, this paper proposes a multi-agent fact-checking framework called FRAME (Faithful, Reproducible, Agent-based, Multi-stage, and Evidence perception). FRAME addresses these limitations through two core designs: firstly, it designs a three-stage faithfulness-gating mechanism, embedding a faithfulness-verifier agent into the pipeline for faithfulness verification, and triggering targeted repairs when unfaithfulness is detected; secondly, it constructs a Bayesian-inspired structured perception framework for evidence-conflict perception, classifying the retrieved evidence, performing multi-attribute reweighting and weighted synthesis, and, finally, outputting a structured decision with a complete reasoning chain. FRAME is systematically evaluated on multiple benchmark datasets, and the experimental results demonstrate the advantages, generalization ability, and fault tolerance of FRAME when dealing with different types of datasets.