Concept

Variational Inference and Learning

In variational inference and learning, inference is framed as an optimization problem where the evidence lower bound (ELBO), denoted as L\mathcal{L}, is maximized with respect to an approximate distribution qq, and learning is achieved by maximizing L\mathcal{L} with respect to the model parameters θ\theta. The equations for the ELBO are L(v,θ,q)=logp(v;θ)DKL(q(hv)p(hv))\mathcal{L}(v,\theta,q)= \log p(v;\theta) - D_{KL}(q(h|v)||p(h|v)) and L(v,θ,q)=Ehq(logp(h,v))+H(q)\mathcal{L}(v,\theta,q)= \mathbb{E}_{h\sim q}(\log p(h,v))+H(q). Because maximizing L\mathcal{L} with respect to θ\theta without restrictions is often intractable, learning is performed over a restricted family of possible distributions for qq. This restricted set is chosen to allow for the tractable computation of Ehq(logp(h,v))\mathbb{E}_{h\sim q}(\log p(h,v)).

0

1

Updated 2026-06-20

References


Tags

Data Science