Learn Before
Concept

Estimate parameters in LDA

First, the means and covariance of the assumed gaussian distributions in each class need to be estimated. When p=1p=1, which means only one predictor, that is to estimate the class specific mean μk\mu_k and the common variance σ2\sigma^2. They can be estimated as follows μ^k=1nki:yi=kxi\hat{\mu}_{k}=\frac{1}{n_{k}} \sum_{i: y_{i}=k} x_{i} σ^2=1nKk=1Ki:yi=k(xiμ^k)2\hat{\sigma}^{2}=\frac{1}{n-K} \sum_{k=1}^{K} \sum_{i: y_{i}=k}\left(x_{i}-\hat{\mu}_{k}\right)^{2} where nn is the total number of training observations, and nkn_k is the number of training observations in the kth class. When p>1p>1, which means multiple predictors, the estimation would be similar but much more complicated with p(p+1)/2p(p+1)/2 parameters in the covariance matrix.

As for the prior probabilty πk\pi_k, it can be estimated based on the proportion of class kk observations in the training set, which would be

π^k=nk/n.\hat{\pi}_{k}=n_{k} / n.

0

2

Updated 2020-02-26

Tags

Data Science