This manipulation consists of replacing the face of one person in a video with the face of another person to generate a realistic fake video. The quality of training dataset has evolved dramatically. Early datasets are of low quality, and poorly synthesized, while the latest datasets are more natural and realistic. The algorithms has also evolved through the last few years. Early algorithms include the use of PCA, Logistic Regression, and SVM. The deep learning approach has enhanced the performance extensively. Algorithms include CNN, RNN, and GAN has been the mainstream in doing Identity Swap. In addition to the algorithms, we should consider the ethics of Identity Swap since it can be used for malicious purposes.