Ratio matching is a technique applied to discrete data, specifically binary data. It avoids evaluating the intractable partition function by incorporating a ratio of probabilities, which leads to the cancellation of the partition function. Ratio matching minimizes the following objective function: $$L^{(RM)}(x,\theta) = \sum_{j=1}^n\left(\frac{1}{1+ \frac{p_{model}(x;\theta)}{p_{model}(f(x,j);\theta)}}\right)^2$$

San Jose State University

Google

There are several methods to train and evaluate undirected graphical models with intractable partition function.

One way is to avoid using partition function and this can be done by following the techniques mentioned below.

 - Log-Likelihood Gradient
 - Stochastic Maximum Likelihood and ContrastiveDivergence
 - Pseudolikelihood
 - Score Matching and Ratio Matching
 - Denoising Score Matching
 - Noise-Contrastive Estimation

Training and Evaluation of Models with Intractable Partition Functions

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Score matching helps to train a neural network model without evaluating the partition function , $Z$ or its derivative.
The strategy used is to minimize the expected squared difference between the derivatives of the model's log density with respect to the input and the derivatives of the data's log density with respect to the input.

$L(x,\theta) = \frac{1}{2} \| \nabla_x \log p_{model}(x;\theta),- \nabla_x \log p_{data}(x) \|_{2}^{2}$

$ J(\theta) = \frac{1}{2} E_{p_{data}}(x)L(x,\theta)$

$\theta^* = \min\limits_{\theta} J(\theta)$


Learn Before

Related