We present a framework for visual action planning of complex manipulation tasks with high-dimensional statespaces, focusing on manipulation of deformable objects. Wepropose a Latent Space Roadmap (LSR) for task planningwhich is a graph-based structure globally capturing the systemdynamics in a low-dimensional latent space. Our frameworkconsists of three parts: (1) a Mapping Module (MM) that mapsobservations given in the form of images into a structuredlatent space extracting the respective states as well as generatesobservations from the latent states, (2) the LSR which builds andconnects clusters containing similar states in order to find thelatent plans between start and goal states extracted by MM, and(3) the Action Proposal Module that complements the latent planfound by the LSR with the corresponding actions. We presenta thorough investigation of our framework on simulated boxstacking and rope/box manipulation tasks, and a folding taskexecuted on a real robot. ; QC 20211109


    Access

    Download


    Export, share and cite



    Title :

    Enabling Visual Action Planning for Object Manipulation through Latent Space Roadmap


    Contributors:

    Type of media :

    Paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Enabling Technologies for Space Exploration Missions: The CIRA-TEDS Program Roadmap Perspectives

    Favaloro, N. / Saccone, G. / Piscitelli, F. et al. | Springer Verlag | 2023


    Space Launch Roadmap

    Hutt, John | AIAA | 2003


    Object and Action Classification with Latent Window Parameters

    Bilen, H. / Namboodiri, V. P. / Gool, L. J. | British Library Online Contents | 2014


    Enabling Exploration - The Technology Development Roadmap in ESA's Aurora Programme

    Cawthorne, Andrew / Gardini, Bruno / Ongaro, Franco et al. | AIAA | 2004


    Space ROS: Architecture Roadmap

    Austin Probe / Amalaye Oyake / Will Chambers | NTRS | 2022