Formula

Bounding Log-Likelihood with Jensen's Inequality

A common application of Jensen's inequality is to bound a more complicated expression by a simpler one, such as the log-likelihood of partially observed random variables. This is utilized in variational methods using the inequality EYP(Y)[logP(XY)]logP(X)E_{Y \sim P(Y)}[-\log P(X \mid Y)] \geq -\log P(X), since P(Y)P(XY)dY=P(X)\int P(Y) P(X \mid Y) dY = P(X). In this context, YY typically represents an unobserved random variable (such as cluster labels in clustering), P(Y)P(Y) is its estimated distribution, and P(XY)P(X \mid Y) is the generative model, with P(X)P(X) being the distribution with YY integrated out.

0

1

Updated 2026-05-15

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L