The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms. © 2015 Elsevier B.V. ; Luiz Celiberto Jr. and Reinaldo Bianchi acknowledge the support of FAPESP (grants 2012/14010-5 and 2011/19280-8). Paulo E. Santos acknowledges support from FAPESP (grant 2012/04089-3) and CNPq (grant PQ2 -303331/2011-9). ; Peer Reviewed


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Transferring knowledge as heuristics in reinforcement learning: A case-based approach



    Erscheinungsdatum :

    2015-01-01



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    006 / 629



    Reinforcement Learning Heuristics for Aerospace Control Systems

    Robinette, Preston K. / Heiner, Benjamin K. / Ravaioli, Umberto et al. | IEEE | 2022


    Learning 2-Opt Heuristics for Routing Problems via Deep Reinforcement Learning

    de O. da Costa, Paulo R. / Rhuggenaath, Jason / Zhang, Yingqian et al. | BASE | 2021

    Freier Zugriff

    Using Cases as Heuristics in Reinforcement Learning: A Transfer Learning Application

    Celiberto, Luiz A. / Matsuura, Jackson P. / López de Mántaras, Ramón et al. | BASE | 2011

    Freier Zugriff

    Heuristics-oriented overtaking decision making for autonomous vehicles using reinforcement learning

    Liu, Teng / Huang, Bing / Deng, Zejian et al. | IET | 2020

    Freier Zugriff

    Transferring knowledge across robots: A risk sensitive approach

    Costante, Gabriele / Ciarfuglia, Thomas A. / Valigi, Paolo et al. | BASE | 2015

    Freier Zugriff