Michigan State University

Unsupervised machine learning methods find structure in unlabeled data.
They are used when we DON'T have prior knowledge about the categories.

Unsupervised statistical learning

Which of the following are use-cases of supervised learning?

Principal component analysis (PCA) is a method of unsupervised learning techniques that functions as a method for dimension reduction in regression models.  

As explained in the textbook, the objective of PCA is *to find a low-dimensional representation of the observations that explain a good fraction of the variance* (ISLR).


Principal Components Analysis (PCA)

Unlike supervised learning, unsupervised learning is used as part of exploratory data analysis (EDA) which aims to depict the main characteristics of a data set. However, it is difficult to use cross-validation in order to check the validity of our results when using unsupervised learning methods. This is because unlike supervised learning, which can check the accuracy of a predictive model by using it on data not used in fitting the model, unsupervised learning does not have any output values to match to, therefore making it difficult to assess the results since we don’t know the ‘true’ answer.

The Challenge of Unsupervised Learning

Types of unsupervised learning problems

Which ones are true about Supervised statistical learning?

Association rules allow you to establish associations amongst data objects inside large databases. This unsupervised technique is about discovering interesting relationships between variables in large databases. For example, people that buy a new home most likely to buy new furniture.

Association

T-SNE is an unsupervised machine learning method that is used to visualize the higher dimensional data in low dimensions. T-SNE is used for designing/implementation and can bring down any number of feature space into 2-D feature space.

T-Distributed Stochastic Neighbour Embedding (T-SNE)

Clustering tries to place unlabeled samples into groups based on similar characteristics. 
An example, these samples can be distinguished into 5 categories by some observed characteristic. Let's say we are clustering textbook samples by different school subjects (math, science, history, english, art). 
![The image from the cited reference](https://firebasestorage.googleapis.com/v0/b/onecademy-1.appspot.com/o/UploadedImages%2FMadeline-Rosenberg_Wed%2C%2019%20Feb%202020%2015%3A27%3A54%20GMT.png?alt=media&token=5b890a1d-803b-4ad5-a65f-1ec4043225c2) 

Clustering, an unsupervised statistical learning method

The primary advantages of unsupervised learning include:
- It identifies unknown patterns within data.
- It discovers features that are useful for categorization.
- It can operate in real-time, allowing input data to be analyzed dynamically.
- It utilizes unlabeled data, which is more abundant and easier to obtain than labeled data since it requires no manual intervention.

Advantages of Unsupervised Learning

This methodology is frequently used in recommendation problems. The examples  include deciding which stories to show on social media feed, which movies to suggest to user and many more.  In simple terms **collaborative filtering** finds what particular user liked and tries to find users with similar preferences and based on this information it would recommend to the current user the items that these other users liked. 

Collaborative Filtering

Independent component analysis (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or signals. ICA defines a generative model for the observed multivariate data, which is typically given as a large database of samples.

Independent Component Analysis

Clustering: Clustering is the process of grouping the given data into different clusters or groups. 

Possible applications are :

1) e-commerce websites using clustering algorithms to implement a user-specific recommendation system.
2) Video recommendation by detecting groups of similar subscribers, 

Commonly used algorithms are:

1)K-Means
2)Expectation Maximization
3)Hierarchical Cluster Analysis (HCA)

(https://www.pythonistaplanet.com/applications-of-unsupervised-learning/)

Real-World Applications Of Unsupervised Learning

A type of linear factor model that uses the slowness principle and applies a cost to time. For features that may not change rapidly over time, e.g. whether or not an animal is in frame of a video, slow feature analysis can smooth the detection of those features. A cost may be added of the following form:
$$\lambda \sum_{t} L(f(x^{(t+1)}), f(x^{(t)}))  $$
It has not been used frequently or in any state-of-the-art applications, but may hold promise for more interesting system to come.  

Slow Feature Analysis

Linear Factor Models Lecture - Berkeley

Slow feature analysis is motivated by a general principle called the slowness principle. The idea is that the important characteristics of scenes change very slowly compared to the individual measurements that make up a description of a scene.  The slowness principal may be introduced by adding a term to the cost function measuring the difference between feature extractors according to time. 
The SFA algorithm consists of defining $f(x; \theta)$ to be a linear transformation and solving the optimization problem:
$min_{\theta}E_t (f(x^{t+1})_i-f(x^{t})_i)^2$
subject to the constraints:
$E_t (f(x^{t})_i)=0$
and
$E_t [f(x^{t})^2_i]=1$

Learn Before

Related