H-GEMA: Hybrid Geo-stochastic Error-Mitigating Architecture for GPU Cluster Scheduling

Abstract

This work presents H-GEMA (Hybrid Geo-stochastic Error-Mitigating Architecture), a five-phase scheduling architecture for heterogeneous GPU clusters designed to balance real-time inference latency, batch/training throughput, topology-aware GPU placement, and starvation prevention. H-GEMA combines a Lyapunov drift-plus-penalty Virtual Queue Tracker for adaptive queue-debt management, an online A-NWIN Whittle-index neural estimator for dynamic task prioritization, a Bertsekas Auction-based assignment mechanism for scalable GPU allocation, topology-aware placement costs, thermal filtering, and final VRAM/thermal safety validation. The architecture is evaluated through tick-based simulation on NVIDIA A100 GPU clusters under four workload scenarios: training bursts, normal mixed operations, constrained capacity, and extreme priority flooding. After correcting implementation defects identified during an independent audit, H-GEMA eliminates real-time SLA violations in the training-burst scenario (0% versus 11.51% for the greedy baseline), reduces topological dispersion by approximately 31–50% in three scenarios, and substantially reduces SLA violations during priority flooding. The experiments also expose an important limitation under sustained mixed workloads: long-duration multi-GPU training jobs can occupy cluster capacity and delay subsequent real-time tasks. The paper therefore identifies duration-aware scheduling, incorporating predicted GPU-time requirements into the priority estimator, as the primary direction for future development. The work is intended as a reproducible architectural and experimental contribution to research on GPU cluster scheduling, stochastic optimization, topology-aware resource allocation, real-time AI infrastructure, and learning-assisted scheduling.