- Being able to back-propagate through this sampling operation allows us to incorporate it into a larger graph. We can build elements of the graph on top of the output of the sampling distribution and elements whose outputs are the inputs or the parameters of the sampling operation. 

New York University

- One straightforward way to extend neural networks to implement stochastic transformations of x is to augment the neural network with extra inputs z that are sampled from some simple probability distribution, such as a uniform or Gaussian distribution. 
- Let us consider the operation consisting of drawing samples y from a Gaussian distribution with mean μ and variance σ^2:
 (refer to equation 20.54 below)
- We can rewrite the sampling process as transforming an underlying random value z ∼ N(z; 0, 1) to obtain a sample from the desired distribution:  (refer to equation 20.55 below)
- We can now back-propagate through the sampling operation, by regarding it as a deterministic operation with an extra input z. 

Learn Before

Related