This paper proposes GraviCap, i.e., a new approach for joint markerless 3D human motion capture and object trajectory estimation from monocular RGB videos. We focus on scenes with objects partially observed during a free flight. In contrast to existing monocular methods, we can recover scale, object trajectories as well as human bone lengths in meters and the ground plane's orientation, thanks to the awareness of the gravity constraining object motions. Our objective function is parametrised by the object's initial velocity and position, gravity direction and focal length, and jointly optimised for one or several free flight episodes. The proposed human-object interaction constraints ensure geometric consistency of the 3D reconstructions and improved physical plausibility of human poses compared to the unconstrained case. We evaluate GraviCap on a new dataset with ground-truth annotations for persons and different objects undergoing free flights. In the experiments, our approach achieves state-of-the-art accuracy in 3D human motion capture on various metrics. We urge the reader to watch our supplementary video. Both the source code and the dataset are released; see http://4dqv.mpi-inf.mpg.de/GraviCap/.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Gravity-Aware Monocular 3D Human-Object Reconstruction ; Gravity-Aware Monocular {3D} Human-Object Reconstruction


    Beteiligte:
    Dabral, R. (Autor:in) / Shimada, S. (Autor:in) / Jain, A. (Autor:in) / Theobalt, C. (Autor:in) / Golyanik, V. (Autor:in)

    Erscheinungsdatum :

    2021-01-01


    Medientyp :

    Paper


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    DDC:    629




    Shape-Aware Monocular 3D Object Detection

    Chen, Wei / Zhao, Jie / Zhao, Wan-Lei et al. | IEEE | 2023


    Occlusion-Aware Plane-Constraints for Monocular 3D Object Detection

    Yao, Hongdou / Chen, Jun / Wang, Zheng et al. | IEEE | 2024


    Monocular 3D Ray-Aware RPN For Roadside View Object Detection

    Zhang, Caiji / Tian, Bin / Sun, Yang et al. | IEEE | 2023



    Instance-Aware Monocular 3D Semantic Scene Completion

    Xiao, Haihong / Xu, Hongbin / Kang, Wenxiong et al. | IEEE | 2024