Reinforcement Learning (RL) methods have demonstrated promising results for the automation of subtasks in surgical robotic systems. Since many trial and error attempts are required to learn the optimal control policy, RL agent training can be performed in simulation and the learned behavior can be then deployed in real environments. In this work, we introduce an open-source simulation environment providing support for position based dynamics soft bodies simulation and state-of-the-art RL methods. We demonstrate the capabilities of the proposed framework by training an RL agent based on Proximal Policy Optimization in fat tissue manipulation for tumor exposure during a nephrectomy procedure. Leveraging on a preliminary optimization of the simulation parameters, we show that our agent is able to learn the task on a virtual replica of the anatomical environment. The learned behavior is robust to changes in the initial end-effector position. Furthermore, we show that the learned policy can be directly deployed on the da Vinci Research Kit, which is able to execute the trajectories generated by the RL agent. The proposed simulation environment represents an essential component for the development of next-generation robotic systems, where the interaction with the deformable anatomical environment is involved.


    Access

    Download


    Export, share and cite



    Title :

    Soft Tissue Simulation Environment to Learn Manipulation Tasks in Autonomous Robotic Surgery



    Publication date :

    2020-01-01



    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Force Feedback Enhancement for Soft Tissue Interaction Tasks in Cooperative Robotic Surgery

    BERETTA, ELISA / NESSI, FEDERICO / FERRIGNO, GIANCARLO et al. | BASE | 2015

    Free access

    Vision Based Robotic Interception in Industrial Manipulation Tasks

    Ahmet Denker / Tuğrul Adıgüzel | BASE | 2007

    Free access

    Design and simulation of autonomous robotic units for collaborative tasks

    Melaseval Chithirainkannu Krisshnan, Balajee Arun | BASE | 2019

    Free access

    Parameter Estimation for Deformable Objects in Robotic Manipulation Tasks

    Millard, David / Preiss, James A. / Barbič, Jernej et al. | Springer Verlag | 2023


    Visual Target Recognition and Tracking for Autonomous Manipulation Tasks

    Naylor, Michael / Atkins, Ella / Roderick, Stephen | AIAA | 2007