3D face modelling using a 3D morphable model

3D Morphable Models are used for face analysis because the intrinsic properties of 3D faces provide a representation that is immune to intra-personal variations such as pose and illumination. Given a single facial input image, a 3DMM can recover 3D face (shape and texture) and scene properties (pose and illumination) via a fitting process.

The Surrey 3D face models

Each of our face models is created from a set of 3D face scans. Each scan is in the form of a graph, where the vertices are locations on the surface of the face, and the edges connect the vertices to form a triangulated mesh. Each vertex also has a colour; hence the vertices define both the shape and the texture of a face. Each face is registered to a standard mesh, so that each vertex has the same location on any registered face.

The model has two components: (i) a mesh consisting of the mean face, and (ii) two matrices, one each for shape and texture that describe the various modes of variations from the mean. The number of modes of variation depends on the size of the mesh, and also is different for shape and texture. Hence the appearance of a given face can be summarised by a set of coefficients that describe how much there is of each mode of variation.

If you would like to download and use any of the University of Surrey 3D face models, details of their availability are here.

Development of 3DMMs for face modelling at the University of Surrey

frame from sequence

The work of our group on 3D Morphable Models followed on from early work of Blanz and Vetter, and also from the PhD thesis of S. Romdhani. The development has taken place in several phases:

  1. The PhD work of Rafael Tena fused that of the above researchers, resulting in the creation of a morphable model built from a collection of 169 high-quality 3D face images. The main software components of this work are:
  2. Pouria Mortazavian extended this method to work with low-resolution images:
  3. The main problem with the above fitting methods is that they use gradient descent optimisation, and hence are slow. Guosheng Hu built on this work to provide a faster method using closed-form optimisation solutions:
  4. The above work concentrated on fitting a 3DMM to a single image. Patrik Huber has extended this work to fit a 3DMM to frames from a video. Suitable frames are automatically extracted from the video, and the model fittings of these fames is combined to provide stronger evidence of identity.

Bill Christmas
Last modified: Thu Jun 16 15:50:31 BST 2016