This paper proposes GraviCap, i.e., a new approach for joint markerless 3D human motion capture and object trajectory estimation from monocular RGB videos. We focus on scenes with objects partially observed during a free flight. In contrast to existing monocular methods, we can recover scale, object trajectories as well as human bone lengths in meters and the ground plane's orientation, thanks to the awareness of the gravity constraining object motions. Our objective function is parametrised by the object's initial velocity and position, gravity direction and focal length, and jointly optimised for one or several free flight episodes. The proposed human-object interaction constraints ensure geometric consistency of the 3D reconstructions and improved physical plausibility of human poses compared to the unconstrained case. We evaluate GraviCap on a new dataset with ground-truth annotations for persons and different objects undergoing free flights. In the experiments, our approach achieves state-of-the-art accuracy in 3D human motion capture on various metrics. We urge the reader to watch our supplementary video. Both the source code and the dataset are released; see http://4dqv.mpi-inf.mpg.de/GraviCap/.


    Access

    Download


    Export, share and cite



    Title :

    Gravity-Aware Monocular 3D Human-Object Reconstruction ; Gravity-Aware Monocular {3D} Human-Object Reconstruction


    Contributors:
    Dabral, R. (author) / Shimada, S. (author) / Jain, A. (author) / Theobalt, C. (author) / Golyanik, V. (author)

    Publication date :

    2021-01-01


    Type of media :

    Paper


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629




    Shape-Aware Monocular 3D Object Detection

    Chen, Wei / Zhao, Jie / Zhao, Wan-Lei et al. | IEEE | 2023


    Occlusion-Aware Plane-Constraints for Monocular 3D Object Detection

    Yao, Hongdou / Chen, Jun / Wang, Zheng et al. | IEEE | 2024


    Monocular 3D Ray-Aware RPN For Roadside View Object Detection

    Zhang, Caiji / Tian, Bin / Sun, Yang et al. | IEEE | 2023



    Instance-Aware Monocular 3D Semantic Scene Completion

    Xiao, Haihong / Xu, Hongbin / Kang, Wenxiong et al. | IEEE | 2024