In this paper we propose to combine three AI techniques to speed up a Reinforcement Learning algorithm in a Transfer Learning problem: Casebased Reasoning, Heuristically Accelerated Reinforcement Learning and Neural Networks. To do so, we propose a new algorithm, called L3, which works in 3 stages: in the first stage, it uses Reinforcement Learning to learn how to perform one task, and stores the optimal policy for this problem as a case-base; in the second stage, it uses a Neural Network to map actions from one domain to actions in the other domain and; in the third stage, it uses the case-base learned in the first stage as heuristics to speed up the learning performance in a related, but different, task. The RL algorithm used in the first phase is the Q-learning and in the third phase is the recently proposed Case-based Heuristically Accelerated Q-learning. A set of empirical evaluations were conducted in transferring the learning between two domains, the Acrobot and the Robocup 3D: the policy learned during the solution of the Acrobot Problem is transferred and used to speed up the learning of stability policies for a humanoid robot in the Robocup 3D simulator. The results show that the use of this algorithm can lead to a significant improvement in the performance of the agent. ; Peer Reviewed


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Using Cases as Heuristics in Reinforcement Learning: A Transfer Learning Application



    Erscheinungsdatum :

    2011-01-01


    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    DDC:    006 / 629



    Reinforcement Learning Heuristics for Aerospace Control Systems

    Robinette, Preston K. / Heiner, Benjamin K. / Ravaioli, Umberto et al. | IEEE | 2022


    Learning 2-Opt Heuristics for Routing Problems via Deep Reinforcement Learning

    de O. da Costa, Paulo R. / Rhuggenaath, Jason / Zhang, Yingqian et al. | BASE | 2021

    Freier Zugriff

    Heuristics-oriented overtaking decision making for autonomous vehicles using reinforcement learning

    Liu, Teng / Huang, Bing / Deng, Zejian et al. | IET | 2020

    Freier Zugriff

    Transferring knowledge as heuristics in reinforcement learning: A case-based approach

    Bianchi, Reinaldo / Celiberto, Luiz Antonio / Santos, Paulo Eduardo et al. | BASE | 2015

    Freier Zugriff

    Learning and Transfer of Movement Gaits Using Reinforcement Learning

    Waidner, David / Strand, Marcus | TIBKAT | 2022