Both PCA and LDA are used for visualization and dimensionality reduction but T-SNE is specifically used for visualization purposes only. It is well suited for the visualization of high-dimensional datasets. Unlike PCA and LDA, T-SNE is a non-linear data visualizer. It means it doesn’t form a linear line to separate the classes or to calculate the variance and it doesn’t use any norm or distance metric to calculate the distance between points.

University of Michigan - Ann Arbor

T-SNE is an unsupervised machine learning method that is used to visualize the higher dimensional data in low dimensions. T-SNE is used for designing/implementation and can bring down any number of feature space into 2-D feature space.

T-Distributed Stochastic Neighbour Embedding (T-SNE)

LDA assumes that the covariance matrix is common to all classes, while QDA assumes class-specific covariance matrices.  Notice that for a $p$-dimensional observation $X$, one covariance needs $p(p+1)/2$ parameters, and then QDA will need $Kp(p+1)/2$ parameters for the covariance matrices. While this makes QDA much more flexible, the much more parameters also bring high variance. QDA is preferred when the training set is large, and so the variance is not a major concern.

Using LDA vs QDA

https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf

Visualizing Data using t-SNE

T-SNE gives the impression that it has classified the data by bringing it to two-dimensions but in reality, it doesn’t reduce the dimensions. It is a visualizer, which tells how each class is distributed and is there any overlap between them. In hyper-dimensional space, the euclidean distance becomes useless. The similarity between the classes in higher dimensions corresponds to the short distance between them in low dimensions. ( the data points of each class becomes very close) .
concept

Idea about T-SNE

The algorithm starts by calculating the probability of similarity of points in high-dimensional space and calculating the probability of similarity of points in the corresponding low-dimensional space. The similarity of points is calculated as the conditional probability that a point A would choose point B as its neighbor if neighbors were picked in proportion to their probability density under a Gaussian (normal distribution) centered at A. It uses T-test from the T-distribution. It then tries to minimize the difference between these conditional probabilities (or similarities) in higher-dimensional and lower-dimensional space for a perfect representation of data points in lower-dimensional space. To measure the minimization of the sum of the difference of conditional probability t-SNE minimizes the sum of Kullback-Leibler divergence of overall data points using a gradient descent method.

Learn Before

Related