University of Michigan - Ann Arbor

Unsupervised machine learning methods find structure in unlabeled data.
They are used when we DON'T have prior knowledge about the categories.

Unsupervised statistical learning

Supervised learning is a type of machine learning problem where the objective is to predict a designated unknown label based on known inputs. This approach requires a training dataset consisting of examples for which the ground truth labels are already known, allowing the model to learn the mapping from inputs to classifications.

Supervised Learning

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Which of the following are use-cases of supervised learning?

Principal component analysis (PCA) is a method of unsupervised learning techniques that functions as a method for dimension reduction in regression models.  

As explained in the textbook, the objective of PCA is *to find a low-dimensional representation of the observations that explain a good fraction of the variance* (ISLR).


Principal Components Analysis (PCA)

Unlike supervised learning, unsupervised learning is used as part of exploratory data analysis (EDA) which aims to depict the main characteristics of a data set. However, it is difficult to use cross-validation in order to check the validity of our results when using unsupervised learning methods. This is because unlike supervised learning, which can check the accuracy of a predictive model by using it on data not used in fitting the model, unsupervised learning does not have any output values to match to, therefore making it difficult to assess the results since we don’t know the ‘true’ answer.

The Challenge of Unsupervised Learning

Types of unsupervised learning problems

Which ones are true about Supervised statistical learning?

Association rules allow you to establish associations amongst data objects inside large databases. This unsupervised technique is about discovering interesting relationships between variables in large databases. For example, people that buy a new home most likely to buy new furniture.

Association

T-SNE is an unsupervised machine learning method that is used to visualize the higher dimensional data in low dimensions. T-SNE is used for designing/implementation and can bring down any number of feature space into 2-D feature space.

T-Distributed Stochastic Neighbour Embedding (T-SNE)

Clustering tries to place unlabeled samples into groups based on similar characteristics. 
An example, these samples can be distinguished into 5 categories by some observed characteristic. Let's say we are clustering textbook samples by different school subjects (math, science, history, english, art). 
![The image from the cited reference](https://firebasestorage.googleapis.com/v0/b/onecademy-1.appspot.com/o/UploadedImages%2FMadeline-Rosenberg_Wed%2C%2019%20Feb%202020%2015%3A27%3A54%20GMT.png?alt=media&token=5b890a1d-803b-4ad5-a65f-1ec4043225c2) 

Clustering, an unsupervised statistical learning method

The primary advantages of unsupervised learning include:
- It identifies unknown patterns within data.
- It discovers features that are useful for categorization.
- It can operate in real-time, allowing input data to be analyzed dynamically.
- It utilizes unlabeled data, which is more abundant and easier to obtain than labeled data since it requires no manual intervention.

Advantages of Unsupervised Learning

This methodology is frequently used in recommendation problems. The examples  include deciding which stories to show on social media feed, which movies to suggest to user and many more.  In simple terms **collaborative filtering** finds what particular user liked and tries to find users with similar preferences and based on this information it would recommend to the current user the items that these other users liked. 

Collaborative Filtering

Independent component analysis (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or signals. ICA defines a generative model for the observed multivariate data, which is typically given as a large database of samples.

Independent Component Analysis

Clustering: Clustering is the process of grouping the given data into different clusters or groups. 

Possible applications are :

1) e-commerce websites using clustering algorithms to implement a user-specific recommendation system.
2) Video recommendation by detecting groups of similar subscribers, 

Commonly used algorithms are:

1)K-Means
2)Expectation Maximization
3)Hierarchical Cluster Analysis (HCA)

(https://www.pythonistaplanet.com/applications-of-unsupervised-learning/)

Real-World Applications Of Unsupervised Learning

A type of linear factor model that uses the slowness principle and applies a cost to time. For features that may not change rapidly over time, e.g. whether or not an animal is in frame of a video, slow feature analysis can smooth the detection of those features. A cost may be added of the following form:
$$\lambda \sum_{t} L(f(x^{(t+1)}), f(x^{(t)}))  $$
It has not been used frequently or in any state-of-the-art applications, but may hold promise for more interesting system to come.  

Slow Feature Analysis

Linear Factor Models Lecture - Berkeley

Slow feature analysis is motivated by a general principle called the slowness principle. The idea is that the important characteristics of scenes change very slowly compared to the individual measurements that make up a description of a scene.  The slowness principal may be introduced by adding a term to the cost function measuring the difference between feature extractors according to time. 
The SFA algorithm consists of defining $f(x; \theta)$ to be a linear transformation and solving the optimization problem:
$min_{\theta}E_t (f(x^{t+1})_i-f(x^{t})_i)^2$
subject to the constraints:
$E_t (f(x^{t})_i)=0$
and
$E_t [f(x^{t})^2_i]=1$

Slow feature analysis

Methods of supervised statistical learning

- Classification Problems
- Regression Problems

Types of supervised learning problems

Use cases of supervised statistical learning

Which of the following are examples of supervised machine learning? Select all that apply.

Categories of supervised learning algorithms

A Basic Supervised Statistical Learning Workflow

- Training set: to train the model
- Validation set (development set): to tune the model parameters
- Test set: to test the performance of the model on unobserved data

Division of dataset in supervised statistical learning

Feature scaling greatly affects which of the following supervised machine learning methods? 

1. Supervised learning allows you to collect data or produce a data output from the previous experience
2. Helps you to optimize performance criteria using experience
3 Supervised machine learning helps you to solve various types of real-world computation problems.

Adavantages of supervised learning

1. Decision boundary might be overtrained if your training set which doesn't have examples that you want to have in a class
2. You need to select lots of good examples from each class while you are training the classifier.
3. Classifying big data can be a real challenge.
4.Training for supervised learning needs a lot of computation time.


Disadvantages of Supervised Learning

1. Before doing anything else, you need to decide what kind of data is to be used as a training set
2. You need to decide the structure of the learned function and learning algorithm.
3. Gathere corresponding outputs either from human experts or from measurements

Best practices for Supervised Learning


The Supervised Learning Workshop: A New, Interactive Approach to Understanding Supervised Learning Algorithms, 2nd Edition Paperback – February 28, 2020
https://www.amazon.com/Supervised-Learning-Workshop-Interactive-Understanding/dp/1800209045/ref=sr_1_1_sspa?dchild=1&keywords=Supervised+Learning&qid=1613836679&sr=8-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUExQzMzWkdMTDlBVldRJmVuY3J5cHRlZElkPUEwMTY1NTYxMUNKMk8zN1YwR0ZZTyZlbmNyeXB0ZWRBZElkPUEwMjIwMDEwMjRUUzk1MllXUU5DSSZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=

The Supervised Learning Workshop: A New, Interactive Approach to Understanding Supervised Learning Algorithms

Sequence models are statistical supervised learning models that predict/classify based on sequence data.

Similar to other supervised statistical models, sequence models are in two types:
- Sequence prediction
- Sequence classification

Sequence Models

The purpose of a statistical learning model is to use a statistical model and the training data to find the function $\hat{f}$ as an estimate of the unknown function $f$ that maps the matrix of input variables $X$ to the outcome vector $Y$, as best as possible.
$$Y = f(X) + \epsilon \approx \hat{f}(X)$$ 

Purpose of supervised statistical learning

In supervised learning, input values represent the set of known features or attributes provided to a model to make predictions. A critical first step in data preprocessing is partitioning the dataset to separate these input columns from the designated target values that the model aims to estimate.

Input Values

In information retrieval, search ranking is a task that involves imposing a rank on a set of items. Instead of simply determining if an item is relevant to a user's query, the objective is to assign a score to each relevant candidate and display the top-rated elements most prominently, utilizing machine learning and behavioral models to obtain query-dependent relevance scores.

Search Ranking

Sequence learning encompasses machine learning problems that require a model to ingest sequences of inputs, emit sequences of outputs, or both. Unlike standard tasks that process a fixed number of inputs independently, sequence learning models maintain context across successive observations, making them essential for handling varying-length data such as video snippets, text, or patient histories over time.

Sequence Learning

In supervised learning, a target value (often called a label) represents the designated output or objective variable that a model is trained to predict. During data preprocessing, the dataset is typically partitioned to isolate the target column from the input values, enabling the model to learn the mapping from features to the desired prediction.

Target Values

In standard supervised learning, we commonly rely on the independent and identically distributed (IID) assumption. This principle dictates that both the training and test data are drawn independently from the exact same underlying probability distribution, denoted as $$P(X,Y)$$. Without this strong assumption that $$P(\cdot) = Q(\cdot)$$ (where $$Q$$ is the test distribution), it would be impossible to justify using patterns learned from the training data to make predictions on new test data.

Learn Before

Related