Directional Pheromone Gradient Observations for Decentralized Multi-Agent Reinforcement Learning in Swarm Drone Search and Rescue

Abstract

Search and rescue (SAR) for disaster response involves quick and efficient exploration of large, uncertain, and dangerous spaces. In this paper, we introduce a swarm-drone SAR approach with a key innovation in the form of directional pheromone gradient observation, where each agent’s RL policy is informed not only by local pheromone levels but also by the directional gradients in victim likelihood and coverage in four spatial cones, along with neighbor density information. These attributes are based on four virtual pheromone layers, each evolving independently and representing coverage history, victim likelihood, environment risk, and communication quality. The system uses a centralized training with decentralized execution (CTDE) MARL approach. Experiments were conducted on a custom-designed 40×40 grid-world environment involving 10 drones and 20 victims whose locations are unknown to all agents at episode start, with 180 static obstacles and 50 static hazard zones, over 5,000 training episodes and 30 independent evaluation runs. The hybrid agent achieved 98.9% area coverage and 93.3% victim detection, exceeding the RL-only baseline by 17.1 and 21.6 percentage points, respectively, across five independent training seeds (pooled n=150 evaluation runs). Welch’s t-test confirmed statistically significant improvements over RL-only for both area coverage (t=18.88, df=168, p<0.001, Cohen’s d=2.18) and victim detection (t=16.49, df=218, p<0.001, Cohen’s d=1.90). Ablation confirms that excluding directional gradient features reduces coverage and victim detection by 16.7 and 21.6 percentage points, respectively, identifying them as the dominant contributors to hybrid performance.