Exact inference can be described as an optimization problem. Thus, approximate inference algorithms can be derived by approximating the underlying optimization. We assume a probabilistic model consisting of observed variables $v$ and latent variables $h$. Sometimes it is too difficult to compute the log-probability ($log p(v;\theta)$) of the observed data, so we can instead compute a lower bound $\mathcal{L}(v, \theta,q)$ on $log p(v;\theta)$, called the evidence lower bound (ELBO), or the negative variational free energy.