This manipulation creates entire non-existent face images, usually through powerful GAN. These techniques achieve astonishing results, generating high-quality facial images with a high level of realism. The figure below shows some examples for entire face synthesis generated using StyleGAN. This manipulation could benefit many different sectors such as the video game and 3D-modelling industries, but it could also be used for harmful applications such as the creation of very realistic fake profiles in social networks in order to generate misinformation.

Entire Face Synthesis

This manipulation consists of replacing the face of one person in a video with the face of another person to generate a realistic fake video. The quality of training dataset has evolved dramatically. Early datasets are of low quality, and poorly synthesized, while the latest datasets are more natural and realistic. 
The algorithms has also evolved through the last few years. Early algorithms include the use of PCA, Logistic Regression, and SVM. The deep learning approach has enhanced the performance extensively. Algorithms include CNN, RNN, and GAN has been the mainstream in doing Identity Swap. 
In addition to the algorithms, we should consider the ethics of Identity Swap since it can be used for malicious purposes.

Identity Swap

This manipulation, also known as face reenactment, consists of modifying the facial expression of the person while maintaining the identity of the person. 
Early methods solve this problem by first modeling the facial frames of the two faces, then adjust the expression parameters frame by frame for the two videos. The more popular techniques include Face2Face and NeuralTextures, which uses GAN to achieve this expression transformation.
We should also be aware of the ethics of using expression swap since it can be used to harm people.

Expression Swap

This manipulation, also known as face editing or face retouching, consists of modifying some attributes of the face such as the colour of the hair or the skin, the gender, the age, adding glasses, etc. Useful algorithms of achieving Attribute Manipulation are usually carried out through GAN such as Invertible Conditional GAN, encoder-decoder architecture, and starGAN. 

One example of this type of manipulation is the popular FaceApp mobile application. Consumers could use this technology to try on a broad range of products such as cosmetics and makeup, glasses, or hairstyles in a virtual environment.

Attribute Manipulation

Facial manipulations can be categorised in four main different groups regarding the level of manipulation from higher to lower. 

Entire Face Synthesis

Identity Swap

Attribute Manipulation

Expression Swap


Johns Hopkins University

State University of New York at Stony Brook

FAKE images and videos including facial information generated by digital manipulation, in particular with DeepFake methods, have become a great public concern recently. The very popular term “DeepFake” is referred to a deep-learning based technique able to create fake videos by swapping the face of a person by the face of another person.

DeepFake

Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64, 131-148.

Learn Before

Related

Learn After