We present a novel task, i.e., animating a target 3D object through the motion of a raw driving sequence. In previous works, extra auxiliary correlations between source and target meshes or intermedia factors are inevitable to capture the motions in the driving sequences. Instead, we introduce AniFormer, a novel Transformer-based architecture, that generates animated 3D sequences by directly taking the raw driving sequences and arbitrary same-type target meshes as inputs. Specifically, we customize the Transformer architecture for 3D animation that generates mesh sequences by integrating styles from target meshes and motions from the driving meshes. Besides, instead of the conventional single regression head in the vanilla Transformer, AniFormer generates multiple frames as outputs to preserve the sequential consistency of the generated meshes. To achieve this, we carefully design a pair of regression constraints, i.e., motion and appearance constraints, that can provide strong regularization on the generated mesh sequences. Our AniFormer achieves high-fidelity, realistic, temporally coherent animated results and outperforms compared start-of-the-art methods on benchmarks of diverse categories. Code is available: https://github.com/mikecheninoulu/AniFormer.


    Access

    Download


    Export, share and cite



    Title :

    AniFormer: Data-driven 3D Animation with Transformer


    Contributors:
    Chen, Haoyou (author) / Tang, Hao (author) / Sebe, Nicu (author) / Zhao, Guoying (author)

    Publication date :

    2021-10-25



    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629




    Video Input Driven Animation (VIDA)

    Sun, M. / Jepson, A. / Fiume, E. et al. | British Library Conference Proceedings | 2003


    Comments on event driven animation

    Gomez, Julian E. | NTRS | 1987


    Video input driven animation (VIDA)

    Sun, / Jepson, / Fiume, | IEEE | 2003


    A deformation transformer for real-time cloth animation

    Feng, WW / Kim, BU / Yu, Y | BASE | 2010

    Free access

    Real-Time Speech-Driven Face Animation

    Hong, P. / Wen, Z. / Huang, T. et al. | British Library Conference Proceedings | 2002