This paper presents a novel framework for Offline Reinforcement Learning (RL) with online fine tuning for Heating Ventilation and Air-conditioning (HVAC) systems. The framework presents a method to do pre-training in a black box model environment, where the black box models are built on data acquired under a traditional control policy. The paper focuses on the application of Underfloor Heating (UFH) with an air-to-water-based heat pump. However, the framework should also generalize to other HVAC control applications. Because Black box methods are used is there little to no commissioning time when applying this framework to other buildings/simulations beyond the one presented in this study. This paper explores and deploys Artificial Neural Network (ANN) based methods to design efficient controllers. Two ANN methods are tested and presented in this paper; a Multilayer Perceptron (MLP) method and a Long Short Term Memory (LSTM) based method. It is found that the LSTM-based method reduces the prediction error by 45% when compared with a MLP model. Additionally, different network architectures are tested. It is found that by creating a new model for each timestep, performance can be improved additionally 19%. By using these models in the framework presented in this paper, it is shown that a Multi-Agent RL algorithm can be deployed without ever performing worse than an industrial controller. Furthermore, it is shown that if building data from a Building Management System (BMS) is available, an RL agent can be deployed which performs close to optimally from the first day of deployment. An optimal control policy reduces the cost of heating by 19.4 % when compared to a traditional control policy in the simulation presented in this paper.
Data-Driven Offline Reinforcement Learning for HVAC-Systems
2022-12-15
Blad , C , Bøgh , S & Kallesøe , C 2022 , ' Data-Driven Offline Reinforcement Learning for HVAC-Systems ' , Energy , vol. 261 , no. Part B , 125290 . https://doi.org/10.1016/j.energy.2022.125290
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
Reinforcement Learning , HVAC , Multi-agent deep reinforcement learning , Building Management System (BMS) , Black-box models , Optimal control , HVAC-systems , Energy optimization , /dk/atira/pure/sustainabledevelopmentgoals/affordable_and_clean_energy , name=SDG 7 - Affordable and Clean Energy , /dk/atira/pure/sustainabledevelopmentgoals/responsible_consumption_and_production , name=SDG 12 - Responsible Consumption and Production
DDC: | 629 |
AUTONOMOUS VEHICLE TESTING OPTIMIZATION USING OFFLINE REINFORCEMENT LEARNING
Europäisches Patentamt | 2024
|