T-SNE gives the impression that it has classified the data by bringing it to two-dimensions but in reality, it doesn’t reduce the dimensions. It is a visualizer, which tells how each class is distributed and is there any overlap between them. In hyper-dimensional space, the euclidean distance becomes useless. The similarity between the classes in higher dimensions corresponds to the short distance between them in low dimensions. ( the data points of each class becomes very close) .
concept

Idea about T-SNE

https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf

Visualizing Data using t-SNE

The algorithm starts by calculating the probability of similarity of points in high-dimensional space and calculating the probability of similarity of points in the corresponding low-dimensional space. The similarity of points is calculated as the conditional probability that a point A would choose point B as its neighbor if neighbors were picked in proportion to their probability density under a Gaussian (normal distribution) centered at A. It uses T-test from the T-distribution. It then tries to minimize the difference between these conditional probabilities (or similarities) in higher-dimensional and lower-dimensional space for a perfect representation of data points in lower-dimensional space. To measure the minimization of the sum of the difference of conditional probability t-SNE minimizes the sum of Kullback-Leibler divergence of overall data points using a gradient descent method.

Overview of working of T-SNE

Both PCA and LDA are used for visualization and dimensionality reduction but T-SNE is specifically used for visualization purposes only. It is well suited for the visualization of high-dimensional datasets. Unlike PCA and LDA, T-SNE is a non-linear data visualizer. It means it doesn’t form a linear line to separate the classes or to calculate the variance and it doesn’t use any norm or distance metric to calculate the distance between points.

T-SNE Vs (PCA and LDA)

T-SNE is an unsupervised machine learning method that is used to visualize the higher dimensional data in low dimensions. T-SNE is used for designing/implementation and can bring down any number of feature space into 2-D feature space.

University of Michigan - Ann Arbor

Unsupervised machine learning methods find structure in unlabeled data.
They are used when we DON'T have prior knowledge about the categories.

Unsupervised statistical learning

Which of the following are use-cases of supervised learning?

Principal component analysis (PCA) is a method of unsupervised learning techniques that functions as a method for dimension reduction in regression models.  

As explained in the textbook, the objective of PCA is *to find a low-dimensional representation of the observations that explain a good fraction of the variance* (ISLR).


Principal Components Analysis (PCA)

Unlike supervised learning, unsupervised learning is used as part of exploratory data analysis (EDA) which aims to depict the main characteristics of a data set. However, it is difficult to use cross-validation in order to check the validity of our results when using unsupervised learning methods. This is because unlike supervised learning, which can check the accuracy of a predictive model by using it on data not used in fitting the model, unsupervised learning does not have any output values to match to, therefore making it difficult to assess the results since we don’t know the ‘true’ answer.

The Challenge of Unsupervised Learning

Types of unsupervised learning problems

Which ones are true about Supervised statistical learning?

Association rules allow you to establish associations amongst data objects inside large databases. This unsupervised technique is about discovering interesting relationships between variables in large databases. For example, people that buy a new home most likely to buy new furniture.

Association

T-Distributed Stochastic Neighbour Embedding (T-SNE)

Clustering tries to place unlabeled samples into groups based on similar characteristics. 
An example, these samples can be distinguished into 5 categories by some observed characteristic. Let's say we are clustering textbook samples by different school subjects (math, science, history, english, art). 
![The image from the cited reference](https://firebasestorage.googleapis.com/v0/b/onecademy-1.appspot.com/o/UploadedImages%2FMadeline-Rosenberg_Wed%2C%2019%20Feb%202020%2015%3A27%3A54%20GMT.png?alt=media&token=5b890a1d-803b-4ad5-a65f-1ec4043225c2) 

Clustering, an unsupervised statistical learning method

The primary advantages of unsupervised learning include:
- It identifies unknown patterns within data.
- It discovers features that are useful for categorization.
- It can operate in real-time, allowing input data to be analyzed dynamically.
- It utilizes unlabeled data, which is more abundant and easier to obtain than labeled data since it requires no manual intervention.

Advantages of Unsupervised Learning

This methodology is frequently used in recommendation problems. The examples  include deciding which stories to show on social media feed, which movies to suggest to user and many more.  In simple terms **collaborative filtering** finds what particular user liked and tries to find users with similar preferences and based on this information it would recommend to the current user the items that these other users liked. 

Collaborative Filtering

Independent component analysis (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or signals. ICA defines a generative model for the observed multivariate data, which is typically given as a large database of samples.

Independent Component Analysis

Clustering: Clustering is the process of grouping the given data into different clusters or groups. 

Possible applications are :

1) e-commerce websites using clustering algorithms to implement a user-specific recommendation system.
2) Video recommendation by detecting groups of similar subscribers, 

Commonly used algorithms are:

1)K-Means
2)Expectation Maximization
3)Hierarchical Cluster Analysis (HCA)

(https://www.pythonistaplanet.com/applications-of-unsupervised-learning/)

Real-World Applications Of Unsupervised Learning

A type of linear factor model that uses the slowness principle and applies a cost to time. For features that may not change rapidly over time, e.g. whether or not an animal is in frame of a video, slow feature analysis can smooth the detection of those features. A cost may be added of the following form:
$$\lambda \sum_{t} L(f(x^{(t+1)}), f(x^{(t)}))  $$
It has not been used frequently or in any state-of-the-art applications, but may hold promise for more interesting system to come.  

Slow Feature Analysis

Linear Factor Models Lecture - Berkeley

Slow feature analysis is motivated by a general principle called the slowness principle. The idea is that the important characteristics of scenes change very slowly compared to the individual measurements that make up a description of a scene.  The slowness principal may be introduced by adding a term to the cost function measuring the difference between feature extractors according to time. 
The SFA algorithm consists of defining $f(x; \theta)$ to be a linear transformation and solving the optimization problem:
$min_{\theta}E_t (f(x^{t+1})_i-f(x^{t})_i)^2$
subject to the constraints:
$E_t (f(x^{t})_i)=0$
and
$E_t [f(x^{t})^2_i]=1$

Learn Before

Related

Learn After