Formula

Bounding Log-Likelihood with Jensen's Inequality

A common application of Jensen's inequality is to bound a more complicated expression by a simpler one, such as the log-likelihood of partially observed random variables. This is utilized in variational methods using the inequality EY∼P(Y)[−log⁡P(X∣Y)]≥−log⁡P(X)E_{Y \sim P(Y)}[-\log P(X \mid Y)] \geq -\log P(X), since ∫P(Y)P(X∣Y)dY=P(X)\int P(Y) P(X \mid Y) dY = P(X). In this context, YY typically represents an unobserved random variable (such as cluster labels in clustering), P(Y)P(Y) is its estimated distribution, and P(X∣Y)P(X \mid Y) is the generative model, with P(X)P(X) being the distribution with YY integrated out.

0

1

Updated 2026-05-15

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L