Many image-based perception tasks can be formulated as detecting, associating and tracking semantic keypoints, e.g., human body pose estimation and tracking. In this work, we present a general framework that jointly detects and forms spatio-temporal keypoint associations in a single stage, making this the first real-time pose detection and tracking algorithm. We present a generic neural network architecture that uses Composite Fields to detect and construct a spatio-temporal pose which is a single, connected graph whose nodes are the semantic keypoints (e.g., a person’s body joints) in multiple frames. For the temporal associations, we introduce the Temporal Composite Association Field (TCAF) which requires an extended network architecture and training method beyond previous Composite Fields. Our experiments show competitive accuracy while being an order of magnitude faster on multiple publicly available datasets such as COCO, CrowdPose and the PoseTrack 2017 and 2018 datasets. We also show that our method generalizes to any class of semantic keypoints such as car and animal parts to provide a holistic perception framework that is well suited for urban mobility such as self-driving cars and delivery robots.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association


    Beteiligte:
    Kreiss, Sven (Autor:in) / Bertoni, Lorenzo (Autor:in) / Alahi, Alexandre (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    2022-08-01


    Format / Umfang :

    4344572 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    MONOCULAR 2D SEMANTIC KEYPOINT DETECTION AND TRACKING

    CHEN HAOFENG / ARJUN BHARGAVA / RARES ANDREI AMBRUS et al. | Europäisches Patentamt | 2023

    Freier Zugriff

    Detection of motion fields under spatio-temporal non-uniform illumination

    Zhang, L. / Sakurai, T. / Miike, H. | British Library Online Contents | 1999


    END-TO-END MONOCULAR 2D SEMANTIC KEYPOINT DETECTOR AND TRACKER LEARNING

    LEE KUAN-HUI / CHEN KUN-HSIN / CHEN HAOFENG et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    Pothole detection using spatio-temporal saliency

    Jang, Dong-Won / Park, Rae-Hong | IET | 2016

    Freier Zugriff

    Pothole detection using spatio‐temporal saliency

    Jang, Dong‐Won / Park, Rae‐Hong | Wiley | 2016

    Freier Zugriff