Synthesis and analysis of human faces using multi-view, multi-illumination image ensembles
Abstract (Summary)Synthesis of realistic human faces is still one of the biggest challenges in computer graphics. Similarly, analysis of human faces in images has been a traditional topic of research in computer vision. Both research areas related to human faces have many practical applications including face modeling, face tracking, face relighting, and face recognition in diverse industry fields such as games, virtual reality, digital photography, biometrics and security. In this dissertation we explore the problem of model-based 3D face synthesis and analysis using various kind of image ensembles including multi-view image silhouettes, sequential video frames, and single photographs. We design and develop various image-model difference metrics that can be used in an optimization framework to recover the optimal model parameters of our 3D face model from the input images. In the first part of this dissertation we present a novel method for 3D face reconstruction from a set or sequence of 2D binary silhouettes. Experiments with a multi-camera rig as well as monocular video sequences demonstrate the advantages of our 3D modeling framework. In order to determine the set of optimal views required for silhouette-based shape reconstruction, we build on our modeling framework and extend it by aggressive pruning of the view-sphere with view clustering and various imaging constraints. In the second part of this dissertation we present a novel framework that acquires the 3D shape, texture, pose and illumination of a face from a single photograph. Using a custom-built face scanning system which is equipped with 16 digital cameras and 146 directional LED light sources, a large-scale dataset was collected, which consists of 3D face scans and light reflection images of a diverse group of human subjects. From this dataset, we derive a novel measurement-based illumination model that implicitly incorporates cast-shadows and specularities. We also propose a novel fitting framework that estimates the parameters of the morphable model by minimizing the distance of the input image to the dynamically changing illumination subspace. We leverage our modeling and fitting methods to solve two challenging problems in computer graphics and computer vision - face relighting and face recognition.
School:The Ohio State University
School Location:USA - Ohio
Source Type:Master's Thesis
Date of Publication:01/01/2005