When modeling categorical data, the maximum likelihood estimate for the probability that a feature $$k$$ takes a specific value $$l$$, denoted as $$\theta_{kl} = p(x_k = l)$$, is given by its empirical frequency in the dataset:

$$\theta_{kl} = \frac{n_{kl}}{N}$$

where $$n_{kl}$$ is the number of times feature $$k$$ takes the value $$l$$, and $$N$$ is the total number of observations.

Google

Maximum likelihood is a fundamental principle in statistics and machine learning used to estimate the parameters of an assumed probability distribution. The core idea is to find the parameter values that make the observed data most probable under the assumed model. This concept forms the theoretical foundation for maximum likelihood estimation (MLE).

Maximum Likelihood

It's an important principal in selecting the model and describing the math formula of a model's loss function. Let $p_{model}(x,\theta)$ be a parametric family of possibility distribution over the same space indexed by $\theta$. The maximum likelihood estimator for $\theta$ is then defuned as 
$\theta_{ML} = argmax_{\theta}p_{model}(X;\theta) $

Learn Before

Related