Normalized mutual information is a variation of the mutual information score, normalized to range from 0, representing no mutual information, to 1, representing perfect correlation. $$NMI(U,V) = \frac{2I(U,V)}{H(U)+H(V)}$$ where $$H$$ represents the entropy. However, this score does not account for chance.

University of Michigan - Ann Arbor

Google

Mutual information is a measure of dependence from information theory that quantifies how much the knowledge of one variable reduces the uncertainty about another. For categorical variables, or continuous variables binned into categories, mutual information can be calculated as: $$I(U,V) = \sum_{i=1}^{|U|} \sum_{j=1}^{|V|} \frac{|U_i \cap V_j |}{N} \log \frac{|U_i \cap V_j |}{|U_i||V_j|}$$ where $$U$$ and $$V$$ represent the input variables, $$U_i$$ and $$V_j$$ represent the categories of the variables, and $$N$$ is the total number of samples.

Mutual Information

 Isabelle Guyon, Alexander Statnikov, Berna Bakir Batu (2019)
Cause Effect Pairs in Machine Learning (The Springer Series on Challenges in Machine Learning)
https://www.amazon.com/Effect-Machine-Learning-Springer-Challenges/dp/3030218090

Cause Effect Pairs in Machine Learning

Normalized Mutual Information

Adjusted mutual information, proposed by Vinh et al., is a variation of mutual information that takes into account the number of samples in each category: $$AMI(U,V) = \frac{I(U,V) - E(I(U,V))}{\frac{1}{2}(H(U)+H(V))-E(I(U,V))}$$ where $$E$$ represents the expected value and $$H$$ represents entropy.

Learn Before

Related