Abstract We present a novel method to transfer speech animation recorded in low resolution videos onto realistic 3D facial models. Unsupervised learning is utilized on a speech video corpus to find underlying manifold of facial configurations. K-means clustering is applied on the low dimensional space to find key speaking-related facial shapes. With a small set of laser scanner captured 3D models related to the clustering centroid, the facial animation in 2D videos is transferred onto 3D shapes. Especially by virtue of a weak perspective projection model, the underlying mandible rotation is recovered from videos and is utilized to drive 3D skull movements. The adaption of a generic skull onto facial models is guided by a 2D image, Tissue Map. With parsimonious data requirements, our system realizes the animation transferring and gains a realistic rendering effect with the underlying anatomical structure.
Vision Based Speech Animation Transferring with Underlying Anatomical Structure
Computer Vision – ACCV 2006 ; 5 ; 591-600
Lecture Notes in Computer Science ; 3851 , 5
2006-01-01
10 pages
Aufsatz/Kapitel (Buch)
Elektronische Ressource
Englisch
Facial Model , Cluster Centroid , Computer Animation , Locally Linear Embedding , Active Appearance Model Computer Science , Computer Imaging, Vision, Pattern Recognition and Graphics , Pattern Recognition , Image Processing and Computer Vision , Artificial Intelligence (incl. Robotics) , Algorithm Analysis and Problem Complexity
Vision Based Speech Animation Transferring with Underlying Anatomical Structure
British Library Conference Proceedings | 2006
|Real-Time Speech-Driven Face Animation
British Library Conference Proceedings | 2002
|Synthesizing Speech Animation By Learning Compact Speech Co-Articulation Models
British Library Conference Proceedings | 2005
|Lipreadable Frame Animation Driven by Speech Parameters
British Library Conference Proceedings | 1994
|