When we compare object manipulation capabilities in humans and contemporary robots, we observe an intriguing dichotomy: On one hand, robots have access to advanced compute capacity and precise models of physics, yet their object manipulation skills are comparatively narrow and brittle. On the other hand, the human understanding of physics is allegedly acquired from experience and exhibits many predictive shortcomings, yet their manipulation skills far exceed any contemporary robot’s. Motivated by this observation, this thesis studies the question how much robotic manipulation can benefit from embracing data-driven, approximate models of physics and poses the hypothesis that a tight integration of intuition and control can unlock sophisticated manipulation behaviour. In particular, three aspects of physical intuition are investigated: (i) high-level intuitions for visual task assessment and their application in object stacking and tool use, (ii) low-level intuitions for rigid-body motions and their application in rearrangement planning and visuomotor control, (iii) the integration of dynamics approximation into control policy learning and its application in structured exploration of an environment. In the first part, we demonstrate the effectiveness of a visual stability classifier in planning and constructing stable stacks of objects with varying geometries. We also employ a similar task classification technique in a goal-reaching task and show that the associated variational latent space induces an affordance manifold which can be traversed to synthesise suitable tools for a given task. In the second part, we demonstrate that the introduction of dynamics modelling into an object-centric latent space facilitates object disentanglement from raw visual training data and allows to generate physically plausible scenes and videos from scratch. Visual dynamics approximation is also used in our novel, goal-conditioned, visuomotor control architecture where it enables zero-shot transfer to unseen object rearrangement ...


    Access

    Download


    Export, share and cite



    Title :

    Learning physical intuition for robotic manipulation


    Contributors:

    Publication date :

    2022-03-17


    Type of media :

    Theses


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Learning robotic cloth manipulation

    Verleysen, Andreas | BASE | 2022

    Free access

    Combining learning and structure for robotic manipulation

    Kloss, Alina / Eberhard Karls Universität Tübingen | TIBKAT | 2020

    Free access

    Modular robotic manipulation

    BRUEMMER DAVID J | European Patent Office | 2019

    Free access

    Learning Stable Normalizing-Flow Control for Robotic Manipulation

    Abdul Khader, Shahbaz / Yin, Hang / Pietro, Falco et al. | BASE

    Free access

    Self-supervised learning of primitive-based robotic manipulation

    Berscheid, Lars | TIBKAT | 2023

    Free access