R-MCTS: Retrospective Offline Correction for Multi-Agent LLM Task Planning

Abstract

We present R-MCTS, a retrospective (offline) correction framework for LLM-based multi-agent embodied task planning: failed trajectories are recorded, a lightweight scorer estimates action quality from recorded data alone, a Monte Carlo Tree Search rooted at the failure point proposes a correction using only the scorer (zero further live-simulator calls), and the correction becomes a Direct Preference Optimisation (DPO) training signal. Our corrected two-agent pipeline achieves 86.7% task success, up from 81.0% (standard tasks) and 76.9% (harder multi-object tasks) for the uncorrected single-agent policy. We compare directly against LLaMAR, re-implemented with a local Llama-3-8B-Instruct backbone under an identical compute budget: on four real tasks from LLaMAR's own published task set, the baseline achieved zero full completions and 22.9% mean partial coverage, against genuine object-identification errors consistent in kind with our own failure taxonomy. We further report three targeted ablation studies and a six-configuration controlled DPO study - five null results at one adapter configuration, one positive result independently replicated under a separate random seed - isolating adapter capacity and training duration, not preference-data volume alone, as the governing factor for measurable retrospective correction.