The PPMI vector models can be used to compute similarities between words. 

For example, we can find the 10 most similar words to any target word $$w$$ by computing the cosines between $$w$$ and each of the $$V − 1$$ other words, sorting, and looking at the top 10.


Application of the PPMI vector models

Positive pointwise mutual information replaces all negative PMI values with zero:
$$PPMI(w, c) = \max \left( \log_2(\frac{P(w, c)}{P(w)P(c)}), 0 \right)$$

Positive Pointwise Mutual Information Representation

The Positive Pointwise Mutual Information (PPMI) Algorithm is an algorithm for weighting term-term matrices.

University of Michigan - Ann Arbor

Pointwise Mutual Information (PMI) is a measure of how often two events occur, compared with what we would expect if they were independent.

An alternative weighting function to tf-idf, positive pointwise mutual information is used for term-term-matrices, when the vector dimensions correspond to words rather than documents.

$(PMI(w,c) =  \log _{2} (P(w,c)/P(w)P(c))$

$(PPMI(w,c) =  \max(log _{2} (P(w,c)/P(w)P(c)),0)$



Pointwise Mutual Information

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Learn Before

Related

Learn After