Concept

Linear Discriminant Analysis (LDA)

Linear discriminant analysis (LDA) is another means of classification similar to logistic regression, but better suited to handle non-binary classifications (i.e. when there are >=3 possible output classes). 

LDA assumes Gaussian distributions for observations in each class, with means being class specific and a covariance common to all classes. Then with the help of Bayes' theorem, the probability for one observation being in each class can be estimated. The class with the highest probability would be the classification result.

In this assumption, the classification is equivalent to maximize the discriminant function

δk(x)=xTΣ1μk12μkTΣ1μk+logπk\delta_{k}(x)=x^{T} \boldsymbol{\Sigma}^{-1} \mu_{k}-\frac{1}{2} \mu_{k}^{T} \boldsymbol{\Sigma}^{-1} \mu_{k}+\log \pi_{k}

where μk{\mu}_{k} is the mean for observations in the kkth class, πk{\pi}_{k} is the prior probability that an observation belongs to the kkth class, and Σ\Sigma is the common covariance matrix. The discriminant function is linear to xx, which is why the method is called linear discriminant analysis.

1

10

Updated 2021-02-20

Tags

Data Science