This manipulation creates entire non-existent face images. The table below summarises the main publicly available databases for research on detection of image manipulation techniques relying on entire face synthesis. Four different databases are of relevance here, all of them based on the same GAN architectures: ProGAN and StyleGAN. It is interesting to remark that each fake image may be characterised by a specific GAN fingerprint just like natural images are identified by a device-based fingerprint. In fact, these fingerprints seem to be dependent not only of the GAN architecture, but also of the different instances of it.

Manipulation Techniques and Public Databases of Entire Face Synthesis

Different studies have recently evaluated the difficulty of detecting whether faces are real of artificially generated. The table below shows a comparison of the most relevant approaches in this area. It is important to remark that in some cases, different evaluation metrics are considered, e.g., Area Under the Curve (AUC) or Equal Error Rate (EER), which complicates the comparison among the studies.

Manipulation Detection of Entire Face Synthesis

This manipulation creates entire non-existent face images, usually through powerful GAN. These techniques achieve astonishing results, generating high-quality facial images with a high level of realism. The figure below shows some examples for entire face synthesis generated using StyleGAN. This manipulation could benefit many different sectors such as the video game and 3D-modelling industries, but it could also be used for harmful applications such as the creation of very realistic fake profiles in social networks in order to generate misinformation.

State University of New York at Stony Brook

Facial manipulations can be categorised in four main different groups regarding the level of manipulation from higher to lower. 

Entire Face Synthesis

Identity Swap

Attribute Manipulation

Expression Swap


TYPES OF FACIAL MANIPULATIONS

Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64, 131-148.

DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection

Entire Face Synthesis

This manipulation consists of replacing the face of one person in a video with the face of another person to generate a realistic fake video. The quality of training dataset has evolved dramatically. Early datasets are of low quality, and poorly synthesized, while the latest datasets are more natural and realistic. 
The algorithms has also evolved through the last few years. Early algorithms include the use of PCA, Logistic Regression, and SVM. The deep learning approach has enhanced the performance extensively. Algorithms include CNN, RNN, and GAN has been the mainstream in doing Identity Swap. 
In addition to the algorithms, we should consider the ethics of Identity Swap since it can be used for malicious purposes.

Identity Swap

This manipulation, also known as face reenactment, consists of modifying the facial expression of the person while maintaining the identity of the person. 
Early methods solve this problem by first modeling the facial frames of the two faces, then adjust the expression parameters frame by frame for the two videos. The more popular techniques include Face2Face and NeuralTextures, which uses GAN to achieve this expression transformation.
We should also be aware of the ethics of using expression swap since it can be used to harm people.

Expression Swap

This manipulation, also known as face editing or face retouching, consists of modifying some attributes of the face such as the colour of the hair or the skin, the gender, the age, adding glasses, etc. Useful algorithms of achieving Attribute Manipulation are usually carried out through GAN such as Invertible Conditional GAN, encoder-decoder architecture, and starGAN. 

One example of this type of manipulation is the popular FaceApp mobile application. Consumers could use this technology to try on a broad range of products such as cosmetics and makeup, glasses, or hairstyles in a virtual environment.

Learn Before

Related

Learn After