Sequential–hierarchical coordination for multi-task enroute air traffic control

Abstract

Enroute air traffic controllers (ATCOs) face two fundamentally distinct yet interrelated operational tasks: macroscopic flow-level planning and flight level assignment for sector-wide traffic optimization over time horizons on the order of tens of minutes and microscopic real-time conflict detection and resolution (CD&R) between individual aircraft pairs. These tasks operate across heterogeneous spatial and temporal scales, resulting in high dimensionality and strong cross-level coupling that complicate automation design and human–autonomy collaboration. To address these challenges, this paper proposes a Sequential–Hierarchical Coordination (SHC) framework that explicitly bridges macroscopic and microscopic decision layers through a spatiotemporal constraint propagation mechanism. Within this architecture, a Sequential Composition Hierarchical Reinforcement Learning (SCHRL) algorithm is developed. At the high level, an enhanced multi-agent deep deterministic policy gradient (MADDPG) algorithm with invalid-action masking is employed for flight profile (i.e., flight level) optimization. At the low level, a soft actor–critic (SAC) algorithm augmented with a priority-based coordination mechanism is designed to handle real-time conflict resolution. To mitigate trust issue commonly associated with black-box learning-based systems, the proposed framework further integrates expert imitation learning and a dynamic intervention response module, enabling closer behavioral alignment between automated decision-making and ATCO operational practices. Simulation experiments using real-world operational data and high-density traffic conditions show that the proposed framework achieves a 97.6% completion rate for control tasks across the tested scenarios, while reducing ATCO temporal decision costs by more than 34% under ultra-high traffic demand. These findings demonstrate robustness, scalability, and practical promise of the SHC framework for human–autonomy collaborative air traffic control in complex enroute environments.