Which of the following are use-cases of supervised learning?

Principal component analysis (PCA) is a method of unsupervised learning techniques that functions as a method for dimension reduction in regression models.  

As explained in the textbook, the objective of PCA is *to find a low-dimensional representation of the observations that explain a good fraction of the variance* (ISLR).


Principal Components Analysis (PCA)

Unlike supervised learning, unsupervised learning is used as part of exploratory data analysis (EDA) which aims to depict the main characteristics of a data set. However, it is difficult to use cross-validation in order to check the validity of our results when using unsupervised learning methods. This is because unlike supervised learning, which can check the accuracy of a predictive model by using it on data not used in fitting the model, unsupervised learning does not have any output values to match to, therefore making it difficult to assess the results since we don’t know the ‘true’ answer.

The Challenge of Unsupervised Learning

Types of unsupervised learning problems

Which ones are true about Supervised statistical learning?

Association rules allow you to establish associations amongst data objects inside large databases. This unsupervised technique is about discovering interesting relationships between variables in large databases. For example, people that buy a new home most likely to buy new furniture.

Association

T-SNE is an unsupervised machine learning method that is used to visualize the higher dimensional data in low dimensions. T-SNE is used for designing/implementation and can bring down any number of feature space into 2-D feature space.

T-Distributed Stochastic Neighbour Embedding (T-SNE)

Clustering tries to place unlabeled samples into groups based on similar characteristics. 
An example, these samples can be distinguished into 5 categories by some observed characteristic. Let's say we are clustering textbook samples by different school subjects (math, science, history, english, art). 
![The image from the cited reference](https://firebasestorage.googleapis.com/v0/b/onecademy-1.appspot.com/o/UploadedImages%2FMadeline-Rosenberg_Wed%2C%2019%20Feb%202020%2015%3A27%3A54%20GMT.png?alt=media&token=5b890a1d-803b-4ad5-a65f-1ec4043225c2) 

Clustering, an unsupervised statistical learning method

The primary advantages of unsupervised learning include:
- It identifies unknown patterns within data.
- It discovers features that are useful for categorization.
- It can operate in real-time, allowing input data to be analyzed dynamically.
- It utilizes unlabeled data, which is more abundant and easier to obtain than labeled data since it requires no manual intervention.

Advantages of Unsupervised Learning

This methodology is frequently used in recommendation problems. The examples  include deciding which stories to show on social media feed, which movies to suggest to user and many more.  In simple terms **collaborative filtering** finds what particular user liked and tries to find users with similar preferences and based on this information it would recommend to the current user the items that these other users liked. 

Collaborative Filtering

Independent component analysis (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or signals. ICA defines a generative model for the observed multivariate data, which is typically given as a large database of samples.

Independent Component Analysis

Clustering: Clustering is the process of grouping the given data into different clusters or groups. 

Possible applications are :

1) e-commerce websites using clustering algorithms to implement a user-specific recommendation system.
2) Video recommendation by detecting groups of similar subscribers, 

Commonly used algorithms are:

1)K-Means
2)Expectation Maximization
3)Hierarchical Cluster Analysis (HCA)

(https://www.pythonistaplanet.com/applications-of-unsupervised-learning/)

Real-World Applications Of Unsupervised Learning

A type of linear factor model that uses the slowness principle and applies a cost to time. For features that may not change rapidly over time, e.g. whether or not an animal is in frame of a video, slow feature analysis can smooth the detection of those features. A cost may be added of the following form:
$$\lambda \sum_{t} L(f(x^{(t+1)}), f(x^{(t)}))  $$
It has not been used frequently or in any state-of-the-art applications, but may hold promise for more interesting system to come.  

Slow Feature Analysis

Linear Factor Models Lecture - Berkeley

Slow feature analysis is motivated by a general principle called the slowness principle. The idea is that the important characteristics of scenes change very slowly compared to the individual measurements that make up a description of a scene.  The slowness principal may be introduced by adding a term to the cost function measuring the difference between feature extractors according to time. 
The SFA algorithm consists of defining $f(x; \theta)$ to be a linear transformation and solving the optimization problem:
$min_{\theta}E_t (f(x^{t+1})_i-f(x^{t})_i)^2$
subject to the constraints:
$E_t (f(x^{t})_i)=0$
and
$E_t [f(x^{t})^2_i]=1$

Slow feature analysis

Unsupervised machine learning methods find structure in unlabeled data.
They are used when we DON'T have prior knowledge about the categories.

University of Michigan - Ann Arbor

- Supervised statistical learning
- Unsupervised statistical learning
- Time series learning
- Semi-Supervised learning

Types of statistical learning

There are several categories for ML algorithms, including:
 - Supervised Learning
 - Semi-supervised Learning
 - Unsupervised Learning
 - Reinforcement Learning
 - Feature Learning (Representation Learning)

The types of machine learning algorithms differ in their approach, the type of data they input and output, and the type of task or problem that they are intended to solve.


Machine Learning Categories

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Unsupervised statistical learning

Semi-supervised learning lies somewhere between supervised and unsupervised learning. It takes small amount of labeled data to start learning and then moves on to huge quantity of unlabaled data to develop further.

Semi Supervised learning

Differences about the Supervised vs Unsupervised Machine learning

Training a model using labeled data and using this model to predict the labels for new data is known as __________.

Modeling the features of an unlabeled dataset to find hidden structure is known as _____.

A time series is a set of measurements of a quantitative variable taken at sequential intervals over a continuous period of time. This approach allows researchers to track trends and ongoing changes within a specific domain, such as recording a factory's weekly worker productivity over an entire year.

Time Series

Supervised learning is a type of machine learning problem where the objective is to predict a designated unknown label based on known inputs. This approach requires a training dataset consisting of examples for which the ground truth labels are already known, allowing the model to learn the mapping from inputs to classifications.

Supervised Learning

In reinforcement learning, the machine goes through trial and error processes where it is rewarded/penalized for the actions it performs. The machine's goal is to maximize the total reward by leveraging previous attempts to make the next decision. The model starts from completely random trials and eventually leads to something with very sophisticated tactics and skills. Compared to supervised/unsupervised learning, reinforcement learning is much more advanced. A reinforcement model will continuously learn, unlike supervised/unsupervised models which all have an endpoint after the training and test data phases.

Reinforcement Learning

- Feature learning, also called representation learning use machine learning to discover not only the mapping from representation to output but also the representation itself. A feature learning algorithm can discover a good set of features for a simple task in minutes, or for a complex task in hours to months. Feature learning can be either supervised or unsupervised.

- Representation of information to be processed has a great importance in day to day life since it eases the task of information processing. For example usage of Arabic numerals instead of Roman numerals for mathematical operations.

- In the context of machine learning an acceptable representation of data makes the future learning tasks easy. The successive learning tasks decide the choice of information representation.



Feature Learning (Representation Learning)

https://www.amazon.com/Master-Algorithm-Ultimate-Learning-Machine-ebook/dp/B012271YB2

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

•	Inductive reasoning 
•	Connectionism
•	Evolutionary Computation 
•	Bayes’ theorem 
•	Analogical modeling 

Key takeaway: Pedro Domingos believes the “Master algorithm” will be a mixture of different schools of machine learning, as each school of thought has strengths and weaknesses that challenge their ability to be applied universally. 


Learn Before

Related

Learn After