Recent advances in large language models have highlighted their potential to encode massive amounts of semantic knowledge for long-term autonomous decision-making, positioning them as a promising solution for powering the cognitive capabilities of future home-assistant robots. However, while large language models can provide high-level decision, there is still no unified paradigm for integrating them with robots’ perception and low-level action. In this paper, we propose a framework centered around a large language model, integrated with visual perception and motion planning modules, to investigate the robotic grasping task. Unlike traditional methods that only focus on generating stable grasps, our proposed approach can handle personalized user instructions and perform tasks more effectively in home scenarios. Our approach integrates existing state-of-the-art models in a simple and effective way, without requiring any fine-tuning, which makes it low-cost and easy to deploy. Experiments on a physical robot system demonstrate the feasibility of our approach.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Decision-Making in Robotic Grasping with Large Language Models


    Additional title:

    Lect.Notes Computer


    Contributors:
    Yang, Huayong (editor) / Liu, Honghai (editor) / Zou, Jun (editor) / Yin, Zhouping (editor) / Liu, Lianqing (editor) / Yang, Geng (editor) / Ouyang, Xiaoping (editor) / Wang, Zhiyong (editor) / Liao, Jianfeng (author) / Zhang, Haoyang (author)

    Conference:

    International Conference on Intelligent Robotics and Applications ; 2023 ; Hangzhou, China July 05, 2023 - July 07, 2023



    Publication date :

    2023-10-16


    Size :

    10 pages




    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English




    Decision-Making in Robotic Grasping with Large Language Models

    Liao, Jianfeng / Zhang, Haoyang / Qian, Haofu et al. | TIBKAT | 2023


    Sensory Interactive Teleoperator Robotic Grasping

    Alark, Keli / Lumia, Ron | NTRS | 1997


    Fundamentals of robotic grasping and fixturing

    Caihua, Xiong / World Scientific (Firm) | TIBKAT | 2007


    COMPLIANT CONCAVE GRIPPER FOR ROBOTIC GRASPING

    DE AVIK / KENNEALLY GAVIN / TOPPING THOMAS | European Patent Office | 2024

    Free access

    Multimodal Perception for Robotic Grasping and Pouring

    Liang, Hongzhuo / Universität Hamburg / Universität Hamburg, Fakultät für Mathematik, Informatik und Naturwissenschaften et al. | TIBKAT | 2022

    Free access