Maximum likelihood from incomplete data via the EM algorithm which is proposed by Dempster, Laird and Rubin.
The link is : https://www.ece.iastate.edu/~namrata/EE527_Spring08/Dempster77.pdf

University of Michigan - Ann Arbor

A Gaussian mixture model (GMM) is a unsupervised learning model which states that all generated data points are derived from a mixture of a finite Gaussian distributions that has no known parameters. The parameters for Gaussian mixture models are derived from an iterative expectation-maximization algorithm. 

Gaussian Mixture Model (GMM) Clustering 

The EM algorithm is an iterative optimization strategy. Since each iteration in its calculation method is divided into two steps, one is the expectation step (E step) and the other is the maximization step (M step), so the algorithm is called EM Algorithm (Expectation-Maximization Algorithm). The EM algorithm was originally to solve the problem of parameter estimation in the case of missing data.

Expectation Maximization Algorithm

The paper proposing EM algorithm

For the Gaussian Mixture Model, we fix the covariance for each cluster as $\sigma^2I$. If we take $\sigma^2->0$. Then update equations converge to doing K-means clustering.

The Relationship between K-Means and GMM

1. We could directly use BIC to approximate the number of components of GMM
2. We could select it based on the results of the split test. The crotch of the performance is the best candidate of the number of the components.

Learn Before

Related