Detail of the student project

List
Topic:3D face model from a video or multiple images of a subject
Department:Katedra kybernetiky
Supervisor:Ing. Jan Čech Ph.D.
Announce as:DP,BP,PMI,SOP,PRO
Description:faces

Having a collection of multiple images or a short video of a subject, it is possible to reconstruct a 3D model of his/her face. Such reconstructed model can be used in computer graphics applications, e.g. new view generation, relighting, face texture transfer, etc. Alternatively, the model can be used for identity recognition, e.g. by constructing a face signature from geometric characteristics or by generating a virtual frontal-view.

There are several approaches in the literature to reconstruct the model of a face. Very good results can be obtained by fitting a parametric 3D morphable [1] model by using stereo images [2]. Alternatively, facial features (nose, mouth corners, eye corners), can be reconstructed automatically using classical structure from motion techniques even in uncalibrated setup. A 3D reconstruction of a face from video has already been investigated, e.g. [3]. There exist a commercial solution, however it requires a non-negligible manual effort, e.g. [4]. Nevertheless our goal is to estimate the model automatically.

We will provide a code for detection the facial landmarks which also simultaneously estimate a pose of the camera (position+orientation) [5]. This will be a starting point for the reconstruction algorithm design. Promising results were already obtained with rather naive methods, so impressive results are expected.

References

[1] V. Blanz and T. Vetter. A morphable model for the synthesis of 3d faces. In SIGGRAPH 1999.
video.
[2] B. Amberg, A. Blake, A. Fitzgibbon. Reconstructing High Quality Face-Surfaces using Model Based Stereo. In ICCV 2007.
[3] Douglas Fidaleo and Gerard Medioni. Model-Assisted 3D Face Reconstruction from Video. In AMFG, 2007.
[4] Looxis Faceworks: video.
[5] Jan Cech, Vojtech Franc, Jiri Matas. A 3D Approach to Facial Landmarks: Detection, Refinement, and Tracking. In Proc. ICPR, 2014.
Realization form:software, technical report
Date:17.05.2019
Responsible person: Petr Pošík