 Caesar, Holger (2019-03-01), A list of papers on Generative Adversarial (Neural) Networks: nightrome/really-awesome-gan, retrieved 2019-03-02

 A list of papers on Generative Adversarial (Neural) Networks: nightrome/really-awesome-gan

GANs can be used to generate art; The Verge wrote in March 2019 that "The images created by GANs have become the defining look of contemporary AI art." GANs can also be used to inpaint photographs or create photos of imaginary fashion models, with no need to hire a model, photographer or makeup artist, or pay for a studio and transportation.

Fashion, art and advertising

In 2018, GANs reached the video game modding community, as a method of up-scaling low-resolution 2D textures in old video games by recreating them in 4k or higher resolutions via image training, and then down-sampling them to fit the game's native resolution (with results resembling the supersampling method of anti-aliasing). With proper training, GANs provide a clearer and sharper 2D texture image magnitudes higher in quality than the original, while fully retaining the original's level of details, colors, etc. Known examples of extensive GAN usage include Final Fantasy VIII, Final Fantasy IX, Resident Evil REmake HD Remaster, and Max Payne

Video games

Concerns have been raised about the potential use of GAN-based human image synthesis for sinister purposes, e.g., to produce fake, possibly incriminating, photographs and videos. GANs can be used to generate unique, realistic profile photos of people who do not exist, in order to automate creation of fake social media profiles.

In 2019 the state of California considered and passed on October 3, 2019 the bill AB-602, which bans the use of human image synthesis technologies to make fake pornography without the consent of the people depicted, and bill AB-730, which prohibits distribution of manipulated videos of a political candidate within 60 days of an election. Both bills were authored by Assembly member Marc Berman and signed by Governor Gavin Newsom. The laws will come into effect in 2020.

DARPA's Media Forensics program studies ways to counteract fake media, including fake media produced using GANs.

Concerns about malicious applications

1. GAN can be used to detect glaucomatous images helping the early diagnosis which is essential to avoid partial or total loss of vision

2. GANs that produce photorealistic images can be used to visualize interior design, industrial design, shoes, bags, and clothing items or items for computer games' scenes.Such networks were reported to be used by Facebook.

3. GANs can reconstruct 3D models of objects from images, and model patterns of motion in video.

4. GANs can be used to age face photographs to show how an individual's appearance might change with age.

5. Relevance feedback on GANs can be used to generate images and replace image search systems.


.......

Miscellaneous applications

GANs can improve astronomical images and simulate gravitational lensing for dark matter research. They were used in 2019 to successfully model the distribution of dark matter in a particular direction in space and to predict the gravitational lensing that will occur.

GANs have been proposed as a fast and accurate way of modeling high energy jet formation  and modeling showers through calorimeters of high-energy physics experiments. GANs have also been trained to accurately approximate bottlenecks in computationally expensive simulations of particle physics experiments. Applications in the context of present and proposed CERN experiments have demonstrated the potential of these methods for accelerating simulation and/or improving simulation fidelity.

Science (applications of GANs)

GAN applications have increased rapidly:

1. Fashion, art and advertising.

2. Science.

3. Video games.

4. Miscellaneous applications.

University of Michigan - Ann Arbor

A generative adversarial network (GAN) is a class of machine learning framework where two neural networks, a generative model and a discriminant model, contest against each other in a "game" (in the form of a zero-sum game, where one agent's gain is another agent's loss). Given a training set, this technique learns to generate new data with the same statistics as the training set. 

The core idea of a GAN is based on the "indirect" training through the discriminator, which itself is also being updated dynamically. This basically means that the generator is not trained to minimize the distance to a specific image, but rather to fool the discriminator and the discriminator is rewarded every time it correctly deduces if a sample is from the model distribution or the data distribution. This enables the model to learn in an unsupervised manner.

Generative adversarial network(GAN)

Goodfellow, Ian; Pouget-Abadie, Jean; Mirza, Mehdi; Xu, Bing; Warde-Farley, David; Ozair, Sherjil; Courville, Aaron; Bengio, Yoshua (2014). Generative Adversarial Networks (PDF). Proceedings of the International Conference on Neural Information Processing Systems (NIPS 2014). pp. 2672–2680.

Generative Adversarial Networks

A known dataset serves as the initial training data for the discriminator, presenting it with samples from the training dataset, until it achieves acceptable accuracy. Typically the generator is seeded with randomized input that is sampled from a predefined latent space (e.g. a multivariate normal distribution). Thereafter, candidates synthesized by the generator are evaluated by the discriminator. Independent backpropagation procedures are applied to both networks so that the generator produces better images, while the discriminator becomes more skilled at flagging synthetic images. The generator is typically a deconvolutional neural network, and the discriminator is a convolutional neural network.

Method of GAN

Applications of GAN

1. Deep Convolutional GANs (DCGANs)
2. Conditional GANs (cGANs)
3. StackGAN
4. InfoGANs
5. Wasserstein GANs(WGAN)

Different types of GANs

Auto-Regressive Networks are a kind of directed probability graph. The conditional probability is described using neural network and chain rule. The key defination of the  Auto-Regressive Network is:
$p(x) =\prod_{i=1}^{n_i} p(x_i|x_{i-1},...,x_1)$

Auto-Regressive Network

Restricted Boltzmann Machines (RBM), Deep Boltzmann Machines (DBM), and variants:
- Undirected graphical models with latent variables instead of directed

Deep Belief Networks (DBN):
- Single undirected layer and multiple directed layers

Score Matching and Noise-contrastive Estimation (NCE):
- Don't approximate or bound log-likelihood

Generative Stochastic Network (GSN):
- Don’t involve defining a probability distribution explicitly; instead trains a generative machine to draw samples from the desired distribution


Related Work to GANs

Advantages:
- No need for Markov chains
- Handle sharp distributions well compared to Markov chain methods
- Only back propagation is used to obtain gradients
- No inference needed during learning

Disadvantages:
- No explicit $p_g(x)$
- Necessary to avoid over-training of generative model without updating discriminative model

Advantages/Disadvantages of Gans

When tested against three different generative models using the same starting dataset and evaluated using log-likelihood estimates, adversarial nets produce acceptable results to stay competitive with existing models. 

Below are some of the challenges/benefits that some generative models encounter while testing:

GANs tested performance

Legend:
- $\bold{z}$: Noise distribution (mapped to $\bold{x}$).
- $\bold x$: Data distribution.
- **Blue**: Discriminative Distribution.
- **Green**: Generative Distribution.
- **Black**: Data Distribution.

The discriminative distribution is repeatedly updated to minimize the difference between our generative distribution and true data distribution (shown in figure left to right) so the generative distribution becomes a more accurate approximation of what the data shows.

Illustration of GAN's

Updating Algorithm for GAN's

- Proposition 1: Optimal Discriminator $D$ for given generator $G$ is $D_g^*=\frac {p_{data}(\bold x)} {p_{data}(\bold x) + p_g(\bold x)}$.
- Theorem 1: The global minimum of virtual training criterion $C(G)$ is found if and only if $p_g=p_{data}$, having value $-\log 4$.
- Proposition 2: $p_g$ converges to $p_{data}$ if: 
1. 	$G$ and $D$ have enough capacity,
2. At each step of algorithm 1 the discriminator is allowed to reach its optimum given $G$,
3. $p_g$ is updated to improve the criterion $\mathbb{E}_{\bold x ~ p_{data}}[\log D_G^*(\bold x)] + \mathbb{E}_{\bold x ~ p_g}[\log(1-D_G^*(\bold x)]$.


Learn Before

Related

Learn After