Learn Before
Introduction of Denoising Autoencoders
The denoising autoencoder (DAE) receives a corrupted data point as input and is trained to predict the original, uncorrupted data point as its output.
Just like the figure below, C(|x) represents a conditional distribution over corrupted samples , given a data sample x. DAE sample a training example x from the training data and sample a corrupted version from C(|x=x) at first. Then use (|x) as a training example to estimate the autoencoder reconstruction distribution (x| ) = (x|h) where h is output of the encoder f( ) and defined by a decoder g(h).
Typically we can simply perform gradient-based approximate minimization on the negative log-likelihood−log (x | h).As long as the encoder is deterministic, the denoising autoencoder is a feedforward network and may be trained with exactly the same techniques as any other feedforward network.
0
1
Tags
Data Science
Related
Introduction of Denoising Autoencoders
Vector Field of Denoising Autoencoders
History of MLPs for Denoising Dates
Training Encoder-Decoder Models with a Denoising Autoencoding Objective
An engineer trains two autoencoder models on a large dataset of clean, high-resolution images. Model A is a standard autoencoder, trained to reconstruct the original images perfectly. Model B is a denoising autoencoder, trained to reconstruct the original clean images from input images that have been intentionally corrupted with random noise (e.g., salt-and-pepper noise). After training, both models are evaluated on their ability to reconstruct a new set of images that have a different, unseen type of corruption (e.g., a slight blur). Based on their training objectives, which model is expected to perform better on this new task, and why?
A key modification to the standard autoencoder training process is the introduction of a 'corruption' step to create a more robust model. Arrange the following steps to accurately describe a single training iteration for this modified approach, which aims to reconstruct an original data point from a noisy version of it.
An autoencoder model is trained on a large dataset of facial images. During each training step, a clean image (
x) is taken, a random rectangular section of it is completely blacked out to create a corrupted version (~x), and the model is tasked with reconstructing the original, clean image (x) from the corrupted input (~x). Which of the following best explains what the model must learn about the data distribution to succeed at this specific task?