Recent advances in large language models have highlighted their potential to encode massive amounts of semantic knowledge for long-term autonomous decision-making, positioning them as a promising solution for powering the cognitive capabilities of future home-assistant robots. However, while large language models can provide high-level decision, there is still no unified paradigm for integrating them with robots’ perception and low-level action. In this paper, we propose a framework centered around a large language model, integrated with visual perception and motion planning modules, to investigate the robotic grasping task. Unlike traditional methods that only focus on generating stable grasps, our proposed approach can handle personalized user instructions and perform tasks more effectively in home scenarios. Our approach integrates existing state-of-the-art models in a simple and effective way, without requiring any fine-tuning, which makes it low-cost and easy to deploy. Experiments on a physical robot system demonstrate the feasibility of our approach.
Decision-Making in Robotic Grasping with Large Language Models
Lect.Notes Computer
International Conference on Intelligent Robotics and Applications ; 2023 ; Hangzhou, China July 05, 2023 - July 07, 2023
16.10.2023
10 pages
Aufsatz/Kapitel (Buch)
Elektronische Ressource
Englisch
Large Language Model , Robotic Grasping , Decision-making Computer Science , Artificial Intelligence , Software Engineering/Programming and Operating Systems , Computer Applications , User Interfaces and Human Computer Interaction , Computer Communication Networks , Special Purpose and Application-Based Systems
Decision-Making in Robotic Grasping with Large Language Models
TIBKAT | 2023
|Sensory Interactive Teleoperator Robotic Grasping
NTRS | 1997
|Fundamentals of robotic grasping and fixturing
TIBKAT | 2007
|