Real-time black-box optimization for dynamic discrete environments using embedded Ising machines

Abstract

Many real-time systems require the optimization of discrete variables. Black-box optimization (BBO) algorithms and multi-armed bandit (MAB) algorithms perform optimization by repeatedly taking actions and observing the corresponding immediate rewards without any prior knowledge. Recently, a BBO method using an Ising machine has been proposed to find the best action represented by a combination of discrete values and maximize the immediate reward in static environments. By contrast, real-time systems operate in dynamic environments and necessitate MAB algorithms that maximize the average reward over repeated trials. Due to the enormous number of actions resulting from the combinatorial nature of discrete optimization, conventional MAB algorithms cannot effectively optimize actions for dynamic, discrete environments. Here, we show a heuristic method to maximize the average reward for dynamic discrete environments by extending the BBO method, in which an Ising machine efficiently explores actions while accounting for interactions between variables and environmental changes. We demonstrate the adaptability to dynamic environments of the proposed method in a wireless communication system with moving users. The authors propose a heuristic multi-armed bandit method for dynamic discrete optimization by extending an Ising machine-based black-box optimization framework. The method explores combinatorial action spaces under changing conditions and is demonstrated on a wireless communication system.