Globally guided multi-agent reinforcement learning with decoupled critics for cooperative UAV path planning

Abstract

Abstract Cooperative path planning for multiple unmanned aerial vehicles (UAVs) requires the simultaneous achievement of target navigation, collision avoidance, and formation coordination in complex obstacle environments. Existing multi-agent reinforcement learning methods often suffer from sparse rewards, local deadlocks, and conflicts between formation maintenance and obstacle avoidance, which can destabilize policy learning. To address these issues, this paper proposes a Globally Guided Decoupled-Critic Multi-Agent Soft Actor-Critic algorithm (GDC-MASAC). RRT $$^*$$ -based global guidance provides reference waypoints for the leader UAV, reducing inefficient exploration and local deadlocks. Three decoupled Critic branches separately evaluate target navigation, formation maintenance, and safety objectives, while an environment-aware formation reward adjustment relaxes formation tracking near obstacles according to the distance to obstacle boundaries. Experiments under different obstacle densities and swarm sizes show that GDC-MASAC improves mission completion, collision avoidance, and formation coordination. In a scenario with 12 obstacles and 3 UAVs, GDC-MASAC achieves a mission success rate of 86%, whereas the baseline methods achieve success rates below 50%. These results demonstrate the effectiveness of GDC-MASAC for cooperative multi-UAV path planning in cluttered environments.