Meta-Reinforcement Learning for building temperature control: Design and experimental validation

Abstract

This paper presents the design and experimental validation of a Meta-Reinforcement Learning approach for temperature control in a real building with not perfectly known thermal dynamics. The proposed architecture consists of three main modules: an encoder, a controller, and an adapter. The encoder projects the building's uncertain parameters into a low-dimensional latent representation that provides contextual information to the downstream controller, enabling it to interpret the current building dynamics and apply the most suitable temperature control strategy to optimize closed-loop performance. The adapter, in turn, enables the controller to transfer this simulation-trained knowledge to the real system, ensuring effective adaptation to the actual building dynamics. The encoder and the controller are implemented as reinforcement learning agents trained in simulation on a control-oriented model across the range of dynamics induced by parameter uncertainties, whereas the adapter is trained with a supervised regression approach to associate measured trajectories with the corresponding latent representation. Experimental results on a real house located at the SYSLAB Risø Campus of the Technical University of Denmark demonstrate that the Meta-Reinforcement Learning approach is feasible in practice and improves energy efficiency after just three days of data collection.