Autonomous robot navigation within the dynamic unknown environment is of crucial significance for mobile robotic applications including robot navigation in last-mile delivery and robot-enabled automated supplies in industrial and hospital delivery applications. Current solutions still suffer from limitations, such as the robot cannot recognize unknown objects in real-time and cannot navigate freely in a dynamic, narrow, and complex environment. We propose a complete software framework for autonomous robot perception and navigation within very dense obstacles and dense human crowds. First, we propose a framework that accurately detects and segments open-world object categories in a zero-shot manner, which overcomes the over-segmentation limitation of the current SAM model. Second, we proposed the distillation strategy to distill the knowledge to segment the free space of the walkway for robot navigation without the label. In the meantime, we design the trimming strategy that works collaboratively with distillation to enable lightweight inference to deploy the neural network on edge devices such as NVIDIA-TX2 or Xavier NX during autonomous navigation. Integrated into the robot navigation system, extensive experiments demonstrate that our proposed framework has achieved superior performance in terms of both accuracy and efficiency in robot scene perception and autonomous robot navigation.


    Access

    Download


    Export, share and cite



    Title :

    Online Robot Navigation and Manipulation with Distilled Vision-Language Models


    Contributors:

    Publication date :

    2024


    Remarks:

    International Conference on Robotics and Automation (ICRA)


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models

    Cai, Hengxing / Dong, Jinhan / Tan, Jingjun et al. | ArXiv | 2025

    Free access

    Visual robot navigation with omnidirectional vision

    Posada, Luis Felipe | TIBKAT | 2019


    Visual robot navigation with omnidirectional vision

    Posada, Luis Felipe | BASE | 2019

    Free access

    Mobile manipulation with the TiaGo robot: navigation and planning.

    Garcia Peroy, Xavier | BASE | 2019

    Free access