Reinforcement learning-based decision support for admission-time inpatient bed allocation

Abstract

Hospital bed allocation affects patient waiting, bed turnover, discharge flow, and queue accumulation. First-come, first-served (FCFS) is transparent but does not account for congestion, queue composition, expected length of stay (LOS), or downstream turnover. This study developed and evaluated a Double Deep Q-Network (DDQN)-based framework for admission-time inpatient bed allocation. The framework used hospital state, waiting-time, and LOS information to rank eligible waiting patients whenever compatible bed capacity became available. A retrospective discrete-event simulation was parameterized using one year of de-identified data comprising 40,302 admissions across 21 departments at a tertiary hospital in South Korea. DDQN was compared with an XGBoost-based myopic policy (XGB), a dynamic waiting and service (WDS) policy, and FCFS over 30 matched 60-day replications. Paired t-tests evaluated the primary comparisons. The primary analysis used the empirical case mix of 38.2% long-stay patients at an arrival rate of 10 patients/h with predicted LOS. Robustness analyses varied arrival rates, case mixes, and LOS inputs. Under the primary condition, DDQN led five of six outcomes. Total discharges were 11,062.9 under DDQN, 10,938.7 under XGB, 10,896.2 under WDS, and 9,798.3 under FCFS. Mean waiting times were 31.57, 40.37, 42.84, and 80.81 h, respectively, corresponding to reductions of 21.8%, 26.3%, and 60.9% versus XGB, WDS, and FCFS. DDQN also produced the smallest waiting-time gap and the fewest short-stay patients remaining in the queue, but not the smallest long-stay queue. Its throughput, waiting-time, and short-stay queue advantages persisted across higher arrival rates and alternative case mixes, although XGB or FCFS achieved smaller gaps under some conditions. Predicted LOS produced results close to actual LOS and generally outperformed historical mean LOS. The framework can serve as a human-supervised prioritization layer that generates ranked lists of eligible patients when compatible bed capacity becomes available. DDQN improved throughput, overall waiting time, and short-stay queue control under the empirical primary condition, although no policy dominated every outcome. Because this was a single-site retrospective simulation, prospective and external validation is required before routine deployment.