Sim-to-real Deep Reinforcement Learning (DRL) has shown promising in subtasks automation for surgical robotic systems, since it allows to safely perform all the trial and error attempts needed to learn the optimal control policy. However, a realistic simulation environment is essential to guarantee direct transfer of the learnt policy from the simulated to the real system. In this work, we introduce UnityFlexML, an open-source framework providing support for soft bodies simulation and state-of-the-art DRL methods. We demonstrate that a DRL agent can be successfully trained within UnityFlexML to manipulate deformable fat tissues for tumor exposure during a nephrectomy procedure. Furthermore, we show that the learned policy can be directly deployed on the da Vinci Research Kit, which is able to execute the trajectories generated by the DRL agent. The proposed framework represents an essential component for the development of autonomous robotic systems, where the interaction with the deformable anatomical environment is involved.


    Access

    Download


    Export, share and cite



    Title :

    UnityFlexML: Training Reinforcement Learning Agents in a Simulated Surgical Environment



    Publication date :

    2020-01-01


    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Virtual agents in a simulated virtual training environment

    Achorn, Brett / Badler, Norman L. | NTRS | 1993


    Simulated autonomous driving using reinforcement learning: a comparative study on unity’s ML-agents framework

    Savid, Yusef / Mahmoudi, Reza / Maskeliūnas, Rytis et al. | BASE | 2023

    Free access