A method is disclosed in which a dataset of video images showing motion is collected (310), a dataset of ego-motion video images is extracted (315), a self-supervised ego-motion model is trained from the ego motion video images (320), moving objects are identified in a scene flow (350) and motion trajectories are automatically extracted based on the identified moving objects. Based on the outputs from a depth estimation model (325), the self-supervised ego motion model (320) and a self-supervised optical flow method (330), it is possible to remove the ego-motion from the optical flow vectors to find independently moving objects. A dense tracking method (355) can be run on the scene flow to automatically extract motion trajectories. A trajectory prediction model can be trained from scene flow and automatically generated motion trajectories. The scene flow and motion trajectories can be used as ground truth data or training data. The trajectories may be those of pedestrians, but also bicyclists, other traffic participants, and indeed any scene agent.


    Access

    Download


    Export, share and cite



    Title :

    Self-supervised learning of scene flow


    Contributors:
    ERWIN KRAFT (author)

    Publication date :

    2023-11-22


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    IPC:    G06T Bilddatenverarbeitung oder Bilddatenerzeugung allgemein , IMAGE DATA PROCESSING OR GENERATION, IN GENERAL / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G01S RADIO DIRECTION-FINDING , Funkpeilung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    Object detection using planar homography and self-supervised scene structure understanding

    AN LE / CHENG YU-TE / KNIEPS OLIVER WILHELM et al. | European Patent Office | 2023

    Free access

    OBJECT DETECTION USING PLANAR HOMOGRAPHY AND SELF-SUPERVISED SCENE STRUCTURE UNDERSTANDING

    AN LE / CHENG YU-TE / KNIEPS OLIVER WILHELM et al. | European Patent Office | 2024

    Free access

    OBJECT DETECTION USING PLANAR HOMOGRAPHY AND SELF-SUPERVISED SCENE STRUCTURE UNDERSTANDING

    AN LE / CHENG YU-TE / KNIEPS OLIVER WILHELM et al. | European Patent Office | 2021

    Free access

    Self-Driving Car Using Supervised Learning

    Collins, Ryan ;Kumar Maurya, Himanshu ;Ragul, Raj S.R. | Trans Tech Publications | 2023