Purpose: Precise placement of needles is a challenge in a number of clinical applications such as brachytherapy or biopsy. Forces acting at the needle cause tissue deformation and needle deflection which in turn may lead to misplacement or injury. Hence, a number of approaches to estimate the forces at the needle have been proposed. Yet, integrating sensors into the needle tip is challenging and a careful calibration is required to obtain good force estimates. Methods: We describe a fiber-optic needle tip force sensor design using a single OCT fiber for measurement. The fiber images the deformation of an epoxy layer placed below the needle tip which results in a stream of 1D depth profiles. We study different deep learning approaches to facilitate calibration between this spatio-temporal image data and the related forces. In particular, we propose a novel convGRU-CNN architecture for simultaneous spatial and temporal data processing. Results: The needle can be adapted to different operating ranges by changing the stiffness of the epoxy layer. Likewise, calibration can be adapted by training the deep learning models. Our novel convGRU-CNN architecture results in the lowest mean absolute error of 1.59±1.3mN and a cross-correlation coefficient of 0.9997 and clearly outperforms the other methods. Ex vivo experiments in human prostate tissue demonstrate the needle’s application. Conclusions: Our OCT-based fiber-optic sensor presents a viable alternative for needle tip force estimation. The results indicate that the rich spatio-temporal information included in the stream of images showing the deformation throughout the epoxy layer can be effectively used by deep learning models. Particularly, we demonstrate that the convGRU-CNN architecture performs favorably, making it a promising approach for other spatio-temporal learning problems.


    Access

    Download


    Export, share and cite



    Title :

    Spatio-temporal deep learning models for tip force estimation during needle insertion



    Publication date :

    2019-09-01


    Remarks:

    2-s2.0-85066612891


    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629 / 000



    Needle tip force estimation by deep learning from raw spectral OCT data

    Gromniak, Martin / Gessert, Nils Thorben / Saathoff, Thore et al. | BASE | 2020

    Free access

    Incorporating congestion patterns into spatio-temporal deep learning algorithms

    Leiser, Neil / Yildirimoglu, Mehmet | Taylor & Francis Verlag | 2021


    Enhancing haptic feedback of subsurfaces during needle insertion

    Antoni, Sven-Thomas / Soltau, Stefan / Beringhoff, Jens et al. | BASE | 2018

    Free access

    A Dynamic Spatio-Temporal Deep Learning Model for Lane-Level Traffic Prediction

    Bao Li / Quan Yang / Jianjiang Chen et al. | DOAJ | 2023

    Free access

    Spatio‐temporal dynamic navigation for electric vehicle charging using deep reinforcement learning

    Ali Can Erüst / Fatma Yıldız Taşcıkaraoğlu | DOAJ | 2024

    Free access