Attention mechanisms have also been applied to further improve the training process of the detection systems. Dang et al. carried out a complete analysis of different types of facial manipulations. They proposed to use attention mechanisms and popular CNN models such as XceptionNet and VGG16. For the entire face synthesis manipulation, the authors achieved a final 100% AUC and around 0.1% EER considering real faces from CelebA, FFHQ, and FaceForensics++ databases and fake images created through ProGAN and StyleGAN approaches. 

State University of New York at Stony Brook

Different studies have recently evaluated the difficulty of detecting whether faces are real of artificially generated. The table below shows a comparison of the most relevant approaches in this area. It is important to remark that in some cases, different evaluation metrics are considered, e.g., Area Under the Curve (AUC) or Equal Error Rate (EER), which complicates the comparison among the studies.

Manipulation Detection of Entire Face Synthesis

Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64, 131-148.

DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection

The authors hypothesised that the colour is markedly different between real camera images and fake synthesis images. They proposed a detection system based on colour features and a linear Support Vector Machine (SVM) for the final classification, achieving a final 70.0% AUC for the best performance when evaluating with the NIST MFC2018 dataset.

Study from McCloskey and Albright. 

Conjectured that monitoring neuron behavior could also serve as an asset in detecting fake faces since layer-by-layer neuron activation patterns may capture more subtle features that are important for the facial manipulation detection system. Their proposed approach, named FakeSpoter, extracted as features neuron coverage behaviors of real and fake faces from deep face recognition systems, and then trained a SVM for the final classification. The authors tested their proposed approach using real faces from CelebA-HQ and FFHQ databases and synthetic faces created through InterFaceGAN and StyleGAN, achieving for the best performance a final 84.7% fake detection accuracy using the FaceNet model.

Study from Wang et al. 

Nataraj et al. proposed a detection system based on a combination of pixel co-occurrence matrices and Convolutional Neural Networks (CNN). Their proposed approach was initially tested through a database of various objects and scenes created through CycleGAN. Besides, the authors performed an interesting analysis to see the robustness of the proposed approach against fake images created through different GAN architectures (CycleGAN vs. StarGAN), with good generalisation results. This detection approach was implemented later on in considering images from the 100K-Faces database, achieving an EER of 12.3% for the best fake detection performance.

Study from Nataraj et al.

Yu et al. proposed an attribution network architecture to map an input image to its corresponding fingerprint image. Therefore, they learned a model fingerprint for each source, such that the correlation index between one image fingerprint and each model fingerprint serves as softmax logit for classification. Their proposed approach was tested using real faces from CelebA database and synthetic faces created through different GAN approaches, achieving a final 99.5% fake detection accuracy for the best performance. However, this approach seemed not to be very robust against unseen simple image perturbation attacks such as noise, blur, cropping or compression, unless the models were re-trained again.

Study from Yu et al. 

Marra et al. performed an interesting study in order to detect unseen types of fake generated data. Concretely, they proposed a multi-task incremental learning detection method in order to detect and classify new types of GAN generated images, without worsening the performance on the previous ones. Two different solutions regarding the position of the classifier were proposed based on the successful algorithm iCaRL for incremental learning: i) Multi-Task MultiClassifier (MTMC), and ii) Multi-Task Single Classifier (MT-SC). Regarding the experimental framework, five different GAN approaches were considered in the study, CycleGAN, ProGAN, Glow, StarGAN, and StyleGAN. Their proposed detection approach, based on the XceptionNet model, achieved promising results being able to correctly detect new GAN generated images.

Study from Marra et al.

Study from Dang et al.

Neves et al. performed an in-depth experimental assessment of this type of facial manipulation considering different state-of-the-art detection systems and experimental conditions, i.e., controlled and in-the-wild scenarios. Four different fake databases were considered: i) 150,000 fake faces collected online11 and based on StyleGAN architecture, ii) the 100K-faces public database, iii) 80,000 synthetic faces generated using ProGAN, and iv) the iFakeFaceDB database, an improved version of previous fake databases in which the GAN-fingerprint information has been removed using the GANprintR approach. In controlled scenarios, they achieved similar results as the best previous studies (EER = 0.02%). However, in more challenging scenarios in which images (real and fake) come from different sources (mismatch of datasets), a high degradation of the fake detection performance is observed. Finally, the results achieved over their public iFakeFaceDB database with an EER = 4.5% for the best fake detectors remark how challenging is iFakeFaceDB even for the most advanced manipulation detection methods.

Study from Neves et al.

Hulzebosch et al. have recently performed an in-depth analysis of this face manipulation considering different scenarios such as cross-model, cross-data, and post-processing. Fake detection approaches were based on the popular Xception network and ForensicTransfer, which is an Autoencoder approach. In general, bad generalisation results were obtained under unseen scenarios.




Study from Hulzebosch et al

Guarnera et al. proposed a fake image detection system based on the analysis of convolutional traces. Features were extracted using the Expectation-Maximization algorithm. Popular classifiers such as k-Nearest Neighbours (k-NN), Support Vector Machines (SVM), and Linear Discriminant Analysis (LDA) were used for the final detection. Their proposed approach was tested using fake images generated through AttGAN, GDWCT, StarGAN, StyleGAN, and StyleGAN2, achieving a final accuracy of 99.81% for the best performance.

Learn Before

Related