3D Face Reconstruction and Animation

Abstract

Shape reconstruction using structured lighting is considered one of the most reliable techniques for recovering object surface. Coded structured light is a technique based on the projection of light patterns on the measured surface. The projected patterns are coded in order to solve the correspondence problem. Each token of light is projected on the scene carrying a label which indicates where it comes from (i.e. this token represents projector image column, row or pixel). When the token is imaged by the camera, this label is read and decoded to obtain an unique match.
Using a calibrated projector-camera pair, a coded pattern is projected on the object and imaged by the camera. Since the pattern is coded, correspondences between image points in camera frame and points of the projected pattern in the projector frame can be easily found. These correspondences are used to triangulate and extract 3D information of the surface.
In this thesis, 3D human faces are reconstructed using structured lighting. Also, a novel projector calibration method is presented. It is based on passive stereo and triangulation. After face reconstruction, Optitrack facial capturing system is used to record facial expression of a real actor. The captured data is used to animate the reconstructed faces.