PURPOSE: Concentric tube robots are composed of multiple concentric, pre-curved, super-elastic, telescopic tubes that are compliant and have a small diameter suitable for interventions that must be minimally invasive like fetal surgery. Combinations of rotation and extension of the tubes can alter the robot's shape but the inverse kinematics are complex to model due to the challenge of incorporating friction and other tube interactions or manufacturing imperfections. We propose a model-free reinforcement learning approach to form the inverse kinematics solution and directly obtain a control policy. METHOD: Three exploration strategies are shown for deep deterministic policy gradient with hindsight experience replay for concentric tube robots in simulation environments. The aim is to overcome the joint to Cartesian sampling bias and be scalable with the number of robotic tubes. To compare strategies, evaluation of the trained policy network to selected Cartesian goals and associated errors are analyzed. The learned control policy is demonstrated with trajectory following tasks. RESULTS: Separation of extension and rotation joints for Gaussian exploration is required to overcome Cartesian sampling bias. Parameter noise and Ornstein-Uhlenbeck were found to be optimal strategies with less than 1 mm error in all simulation environments. Various trajectories can be followed with the optimal exploration strategy learned policy at high joint extension values. Our inverse kinematics solver in evaluation has 0.44 mm extension and [Formula: see text] rotation error. CONCLUSION: We demonstrate the feasibility of effective model-free control for concentric tube robots. Directly using the control policy, arbitrary trajectories can be followed and this is an important step towards overcoming the challenge of concentric tube robot control for clinical use in minimally invasive interventions.


    Access

    Download


    Export, share and cite



    Title :

    Investigating exploration for deep reinforcement learning of concentric tube robot control


    Contributors:
    Iyengar, K (author) / Dwyer, G (author) / Stoyanov, D (author)

    Publication date :

    2020-06-06


    Remarks:

    International Journal of Computer Assisted Radiology and Surgery , 15 pp. 1157-1165. (2020)


    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Deep Reinforcement Learning for Concentric Tube Robot Path Following

    Iyengar, Keshav / Spurgeon, Sarah / Stoyanov, Danail | BASE | 2023

    Free access


    A Deep Reinforcement Learning Approach for Inverse Kinematics of Concentric Tube Robots

    Iyengar, K / Dwyer, G / Stoyanov, D | BASE | 2019

    Free access

    Sim2Real Transfer of Reinforcement Learning for Concentric Tube Robots

    Iyengar, Keshav / Sadati, SM Hadi / Bergeles, Christos et al. | BASE | 2023

    Free access

    Coordinated Multi-Robot Exploration using Reinforcement Learning

    Mete, Atharva / Mouhoub, Malek / Farid, Ali Moltajaei | IEEE | 2023