Traditional model-based reinforcement learning approaches learn a model of the environment dynamics without explicitly considering how it will be used by the agent. In the presence of misspecified model classes, this can lead to poor estimates, as some relevant available information is ignored. In this paper, we introduce a novel model-based policy search approach that exploits the knowledge of the current agent policy to learn an approximate transition model, focusing on the portions of the environment that are most relevant for policy improvement. We leverage a weighting scheme, derived from the minimization of the error on the model-based policy gradient estimator, in order to define a suitable objective function that is optimized for learning the approximate transition model. Then, we integrate this procedure into a batch policy improvement algorithm, named Gradient-Aware Model-based Policy Search (GAMPS), which iteratively learns a transition model and uses it, together with the collected trajectories, to compute the new policy parameters. Finally, we empirically validate GAMPS on benchmark domains analyzing and discussing its properties.


    Access

    Download


    Export, share and cite



    Title :

    Gradient-Aware Model-Based Policy Search



    Publication date :

    2020-01-01



    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629



    Diagnostic Evaluation of Policy-Gradient-Based Ranking

    Yu, Hai-Tao / Huang, Degen / Ren, Fuji et al. | BASE | 2022

    Free access

    Developing Flight Control Policy Using Deep Deterministic Policy Gradient

    Tsourdos, Antonios / Dharma Permana, Ir. Adhi / Budiarti, Dewi H. et al. | IEEE | 2019


    Anti-drone Policy Learning Based on Self-attention Multi-agent Deterministic Policy Gradient

    Jiang, Tianrui / Zhuang, Dongye / Xie, Haibin | Springer Verlag | 2022


    Anti-drone Policy Learning Based on Self-attention Multi-agent Deterministic Policy Gradient

    Jiang, Tianrui / Zhuang, Dongye / Xie, Haibin | British Library Conference Proceedings | 2022


    Robust Control Strategy for Quadrotor Drone Using Reference Model-Based Deep Deterministic Policy Gradient

    Hongxun Liu / Satoshi Suzuki / Wei Wang et al. | DOAJ | 2022

    Free access