This paper presents a novel framework for Offline Reinforcement Learning (RL) with online fine tuning for Heating Ventilation and Air-conditioning (HVAC) systems. The framework presents a method to do pre-training in a black box model environment, where the black box models are built on data acquired under a traditional control policy. The paper focuses on the application of Underfloor Heating (UFH) with an air-to-water-based heat pump. However, the framework should also generalize to other HVAC control applications. Because Black box methods are used is there little to no commissioning time when applying this framework to other buildings/simulations beyond the one presented in this study. This paper explores and deploys Artificial Neural Network (ANN) based methods to design efficient controllers. Two ANN methods are tested and presented in this paper; a Multilayer Perceptron (MLP) method and a Long Short Term Memory (LSTM) based method. It is found that the LSTM-based method reduces the prediction error by 45% when compared with a MLP model. Additionally, different network architectures are tested. It is found that by creating a new model for each timestep, performance can be improved additionally 19%. By using these models in the framework presented in this paper, it is shown that a Multi-Agent RL algorithm can be deployed without ever performing worse than an industrial controller. Furthermore, it is shown that if building data from a Building Management System (BMS) is available, an RL agent can be deployed which performs close to optimally from the first day of deployment. An optimal control policy reduces the cost of heating by 19.4 % when compared to a traditional control policy in the simulation presented in this paper.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Data-Driven Offline Reinforcement Learning for HVAC-Systems


    Beteiligte:
    Blad, Christian (Autor:in) / Bøgh, Simon (Autor:in) / Kallesøe, Carsten (Autor:in)

    Erscheinungsdatum :

    2022-12-15


    Anmerkungen:

    Blad , C , Bøgh , S & Kallesøe , C 2022 , ' Data-Driven Offline Reinforcement Learning for HVAC-Systems ' , Energy , vol. 261 , no. Part B , 125290 . https://doi.org/10.1016/j.energy.2022.125290



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    629



    Runway Configuration Management with Offline Reinforcement Learning

    Milad Memarzadeh / Tejas Puranik / Jarrett Battistini et al. | NTRS


    Offline Reinforcement Learning for Autonomous Driving with Real World Driving Data

    Fang, Xing / Zhang, Qichao / Gao, Yinfeng et al. | IEEE | 2022


    AUTONOMOUS VEHICLE TESTING OPTIMIZATION USING OFFLINE REINFORCEMENT LEARNING

    LEE RITCHIE | Europäisches Patentamt | 2024

    Freier Zugriff

    Airport Runway Configuration Management with Offline Model-free Reinforcement Learning

    Milad Memarzadeh / Tejas Puranik / Krishna Kalyanam et al. | NTRS


    Airport Runway Configuration Management with Offline Model-free Reinforcement Learning

    Milad Memarzadeh / Tejas Puranik / Krishna Kalyanam et al. | NTRS