Which of the following are use-cases of supervised learning?

Methods of supervised statistical learning

- Classification Problems
- Regression Problems

Types of supervised learning problems

Use cases of supervised statistical learning

Which ones are true about Supervised statistical learning?

Which of the following are examples of supervised machine learning? Select all that apply.

Categories of supervised learning algorithms

A Basic Supervised Statistical Learning Workflow

- Training set: to train the model
- Validation set (development set): to tune the model parameters
- Test set: to test the performance of the model on unobserved data

Division of dataset in supervised statistical learning

Feature scaling greatly affects which of the following supervised machine learning methods? 

1. Decision boundary might be overtrained if your training set which doesn't have examples that you want to have in a class
2. You need to select lots of good examples from each class while you are training the classifier.
3. Classifying big data can be a real challenge.
4.Training for supervised learning needs a lot of computation time.


Disadvantages of Supervised Learning

1. Before doing anything else, you need to decide what kind of data is to be used as a training set
2. You need to decide the structure of the learned function and learning algorithm.
3. Gathere corresponding outputs either from human experts or from measurements

Best practices for Supervised Learning


The Supervised Learning Workshop: A New, Interactive Approach to Understanding Supervised Learning Algorithms, 2nd Edition Paperback – February 28, 2020
https://www.amazon.com/Supervised-Learning-Workshop-Interactive-Understanding/dp/1800209045/ref=sr_1_1_sspa?dchild=1&keywords=Supervised+Learning&qid=1613836679&sr=8-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUExQzMzWkdMTDlBVldRJmVuY3J5cHRlZElkPUEwMTY1NTYxMUNKMk8zN1YwR0ZZTyZlbmNyeXB0ZWRBZElkPUEwMjIwMDEwMjRUUzk1MllXUU5DSSZ3aWRnZXROYW1lPXNwX2F0ZiZhY3Rpb249Y2xpY2tSZWRpcmVjdCZkb05vdExvZ0NsaWNrPXRydWU=

The Supervised Learning Workshop: A New, Interactive Approach to Understanding Supervised Learning Algorithms

Sequence models are statistical supervised learning functions designed to process, predict, or classify based on sequence data. A fundamental task in this domain is to estimate the joint probability of an entire sequence. When dealing with natural language data composed of discrete tokens, such as words, these estimated functions are commonly referred to as language models. Sequence models provide the capacity to evaluate the likelihood of sequences (for example, comparing the naturalness of outputs from translation systems), sample new sequences, and optimize for the most likely sequence outputs.

Sequence Models

The purpose of a statistical learning model is to use a statistical model and the training data to find the function $\hat{f}$ as an estimate of the unknown function $f$ that maps the matrix of input variables $X$ to the outcome vector $Y$, as best as possible.
$$Y = f(X) + \epsilon \approx \hat{f}(X)$$ 

Purpose of supervised statistical learning

In supervised learning, input values represent the set of known features or attributes provided to a model to make predictions. A critical first step in data preprocessing is partitioning the dataset to separate these input columns from the designated target values that the model aims to estimate.

Input Values

In information retrieval, search ranking is a task that involves imposing a rank on a set of items. Instead of simply determining if an item is relevant to a user's query, the objective is to assign a score to each relevant candidate and display the top-rated elements most prominently, utilizing machine learning and behavioral models to obtain query-dependent relevance scores.

Search Ranking

Sequence learning encompasses machine learning problems that require a model to ingest sequences of inputs, emit sequences of outputs, or both. Unlike standard tasks that process a fixed number of inputs independently, sequence learning models maintain context across successive observations, making them essential for handling varying-length data such as video snippets, text, or patient histories over time.

Sequence Learning

In supervised learning, a target value (often called a label) represents the designated output or objective variable that a model is trained to predict. During data preprocessing, the dataset is typically partitioned to isolate the target column from the input values, enabling the model to learn the mapping from features to the desired prediction.

Target Values

In standard supervised learning, we commonly rely on the independent and identically distributed (IID) assumption. This principle dictates that both the training and test data are drawn independently from the exact same underlying probability distribution, denoted as $$P(X,Y)$$. Without this strong assumption that $$P(\cdot) = Q(\cdot)$$ (where $$Q$$ is the test distribution), it would be impossible to justify using patterns learned from the training data to make predictions on new test data.

Independent and Identically Distributed (IID) Assumption

In many traditional supervised learning problems, the target y was usually a single numeric value, either an integer or a real number.

Typical Outputs in Early Supervised Learning

Supervised learning includes linear regression, logistic regression, and neural networks.

Examples of Supervised Learning Methods

Supervised learning offers several advantages: (1) it learns a mapping from inputs to outputs using labeled examples drawn from prior experience, so predictions are grounded in observed data; (2) because the correct outputs are known during training, model performance can be measured and optimized directly against these known labels; (3) this makes supervised learning applicable to a wide range of real-world prediction and classification problems, such as those addressed by linear regression, logistic regression, and neural networks.

Advantages of Supervised Learning

Supervised learning is a method for learning a mapping from inputs x to outputs y from labeled examples (x, y). Common supervised learning algorithms include linear regression, logistic regression, and neural networks.

University of Michigan - Ann Arbor

Claude

Google

- Supervised statistical learning
- Unsupervised statistical learning
- Time series learning
- Semi-Supervised learning

Types of statistical learning

There are several categories for ML algorithms, including:
 - Supervised Learning
 - Semi-supervised Learning
 - Unsupervised Learning
 - Reinforcement Learning
 - Feature Learning (Representation Learning)

The types of machine learning algorithms differ in their approach, the type of data they input and output, and the type of task or problem that they are intended to solve.


Machine Learning Categories

A dataset is collections of data, which can be in the form of a table, network, text, image, video, sound, ...

Dataset

Machine learning is a branch of artificial intelligence focused on building systems that improve their task performance by learning from data or other experience. It draws on ideas from statistics, optimization, and data analysis to make predictions, recognize patterns, and support decisions.

Machine Learning

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Dive into Deep Learning

Unsupervised machine learning methods find structure in unlabeled data.
They are used when we DON'T have prior knowledge about the categories.

Unsupervised statistical learning

Semi-supervised learning lies somewhere between supervised and unsupervised learning. It takes small amount of labeled data to start learning and then moves on to huge quantity of unlabaled data to develop further.

Semi Supervised learning

Differences about the Supervised vs Unsupervised Machine learning

Training a model using labeled data and using this model to predict the labels for new data is known as __________.

Modeling the features of an unlabeled dataset to find hidden structure is known as _____.

A time series is a set of measurements of a quantitative variable taken at sequential intervals over a continuous period of time. This approach allows researchers to track trends and ongoing changes within a specific domain, such as recording a factory's weekly worker productivity over an entire year.

Time Series

Supervised Learning

This learning approach trains an agent by letting it interact with an environment and observe the consequences of its actions. After each action, the agent receives feedback in the form of rewards or penalties, then adjusts its behavior to improve future outcomes. The objective is to choose actions that maximize long-term reward, not just the next immediate payoff. This is especially useful for tasks where decisions affect later results, such as game playing, robot control, or scheduling.

Learning from Rewards and Penalties

- Feature learning, also called representation learning use machine learning to discover not only the mapping from representation to output but also the representation itself. A feature learning algorithm can discover a good set of features for a simple task in minutes, or for a complex task in hours to months. Feature learning can be either supervised or unsupervised.

- Representation of information to be processed has a great importance in day to day life since it eases the task of information processing. For example usage of Arabic numerals instead of Roman numerals for mathematical operations.

- In the context of machine learning an acceptable representation of data makes the future learning tasks easy. The successive learning tasks decide the choice of information representation.



Feature Learning (Representation Learning)

https://www.amazon.com/Master-Algorithm-Ultimate-Learning-Machine-ebook/dp/B012271YB2

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

•	Inductive reasoning 
•	Connectionism
•	Evolutionary Computation 
•	Bayes’ theorem 
•	Analogical modeling 

Key takeaway: Pedro Domingos believes the “Master algorithm” will be a mixture of different schools of machine learning, as each school of thought has strengths and weaknesses that challenge their ability to be applied universally. 


Machine learning schools of thought (as explained in ”The Master Algorithm” by Pedro Domingos):

What are the categories of machine learning algorithms?

- Number of data points (instances)
- Attributes (variables)
- Types of attributes (variables)
- Number of attributes (variables)
- Statistical measures

Characteristics of a dataset

Here are a list of free resources to find datasets. Most will be available in CSV format for easy integration into R or Python. 

Sample Datasets

- 1) Equilibrium
- 2) Cyclic (periodic)
- 3) Random 
- 4) Complex

Wolfram's four classes of empirical data

Data can be distributed in several ways. Some of these include:
- Normal (Gaussian) distributions
- Uniform distributions
- Exponential distributions

Data Distributions

A machine learning model is the computational machinery designed to ingest data of a specific type, apply mathematical transformations, and output predictions. In modern applications, the focus is on statistical models whose internal transformations can be estimated and refined using a dataset.

Machine Learning Model

Before data can be fed into any machine learning model, its overall quality must be evaluated and improved. Real-world datasets are often plagued by flaws such as extreme outliers, faulty sensor measurements, and human recording errors. Addressing these discrepancies is an essential part of the data preparation process to ensure the model learns from reliable information.

Data Quality

In real-world machine learning scenarios, datasets are often not provided in a single, simple CSV file. Instead, they may be stored in a relational database, where information is distributed across multiple tables. For example, an e-commerce application might store customer addresses in one table and their purchase data in another, requiring practitioners to extract and merge these separate files before modeling.

Relational Database

Beyond standard categorical and numeric variables, deep learning practitioners frequently work with myriad unstructured data formats. These complex data types include text strings for natural language processing, images and point clouds for computer vision, and audio data. Handling these diverse formats requires specialized approaches and advanced tools compared to traditional tabular data.

Deep Learning Data Types

A data processing bottleneck occurs when the time taken to load, transform, and prepare complex data—such as high-resolution images or large text corpora—exceeds the time required for actual model training. To prevent data preprocessing from severely slowing down the overall machine learning pipeline, practitioners must implement efficient algorithms and utilize advanced tools.

Data Processing Bottleneck

The success of a machine learning model heavily depends on having the right data, as encapsulated by the principle 'garbage in, garbage out.' If a dataset contains mistakes, features that do not predict the target, or reflects societal prejudices by under-representing certain groups, the learning process will fail or produce biased, harmful results.

Machine Learning Dataset Quality

A machine learning example, interchangeably referred to as a data point, sample, or instance, represents an individual record within a dataset. To be useful for learning algorithms, each example is characterized by a specific numerical representation, forming the basis for a model's predictions.

Machine Learning Example

Comma-separated values (CSV) files are a ubiquitous format for storing tabular, spreadsheet-like data. In a CSV file, each line represents a distinct data record, and the individual fields within that record are separated by commas.

CSV File

A data batch, commonly referred to as a minibatch, is a discrete, smaller subset of a larger dataset that is processed simultaneously during a single iteration of machine learning model training or evaluation. Training typically requires multiple passes over a dataset, grabbing one minibatch at a time. Each minibatch consists of a tuple of features and corresponding labels. Processing data in minibatches rather than the entire dataset balances computational efficiency with hardware memory constraints.

Data Batch

When loading datasets for machine learning, the order in which data is read depends on the current mode. For training, data is typically read in a random order (shuffled) to properly update the model. For validation, data is read in a pre-defined, sequential order, which is important for debugging purposes and consistent evaluation.

Training vs. Validation Data Reading Order

The Airfoil Self-Noise dataset is a benchmark dataset originally developed by NASA to study aerodynamic noise produced by different aircraft wing sections. Hosted in the UCI Machine Learning Repository, it is commonly used in deep learning optimization literature for comparing different optimization algorithms.

Airfoil Self-Noise Dataset

Before using a dataset for a machine learning task, it is important to examine it first. This initial inspection helps you (1) understand how much data is missing, (2) judge whether the features need further cleaning, and (3) decide which techniques are appropriate for analyzing the data.

Examining a Dataset Before Machine Learning

Representational learning is a class of machine learning that focuses on automatically discovering the most appropriate way to represent data. By learning these representations directly from the data, it eliminates the need for manual feature engineering and allows models to effectively process and understand complex inputs.

Representational Learning

Kaggle is a popular online platform that hosts machine learning competitions. Each competition centers on a specific dataset, and many are sponsored by stakeholders who offer prizes for the winning solutions. The platform enables direct quantitative comparisons of competing approaches and fosters collaboration by allowing users to interact via forums and share code.

Kaggle Platform

Combining classical machine learning and deep learning within predictive analytics platforms enables users to forecast future trends and behaviors with high precision. These data-driven insights accelerate time-to-value across industries by optimizing courses of action and reducing margins of error, effectively replacing retrospective interpretations with forward-looking decision-making.

Predictive Analytics for Accelerated Decision-Making

A development set, sometimes called a holdout validation set, is a data split used to tune hyperparameters, compare feature choices, and make other model-design decisions during development.

Development Set

A test set is used to estimate how well a trained system performs, but it should not be used to choose the model, features, or hyperparameters.

Purpose of a Test Set

When a project must satisfy more than one evaluation criterion, a common strategy is to set one metric as a minimum acceptable requirement and then optimize a different metric among the solutions that meet that requirement. For example, a team might require inference time to stay under a limit and then select the model with the highest accuracy.

Combining Metrics With a Threshold and an Objective

For a brand-new machine learning task, it is usually very hard to know ahead of time which method will work best. Even skilled researchers often test many different ideas before they find one that performs well enough.

It Is Hard to Predict the Best Machine Learning Strategy Before Trying It

Machine learning development usually proceeds in cycles. A practical loop is to propose an approach, implement it, run an experiment to measure the result, and then use what you learned to choose the next change. Because each round reveals new information, teams often test many ideas before settling on one that works well.

An Iterative Machine Learning Workflow

In machine learning, two broad causes of poor performance are bias and variance. Separating them helps you choose the right next step, such as changing the model, gathering more data, or adjusting the training approach.

Bias and Variance as Two Major Sources of Error

With enough labeled input-output examples, an end-to-end model can learn to map inputs directly to outputs that are more complex than a single number. Those outputs can include text, images, audio, or other structured results.

What End-to-End Learning Can Predict

A pipeline can be poorly designed if the full system performs much worse than human level even though each stage is close to human level on the information it receives. In that case, the weakness is often that important information is being lost, so the design should be reconsidered.

When a Pipeline Is Missing Information

Text polarity detection is the task of deciding whether a piece of text expresses a favorable or unfavorable opinion, such as determining whether a product review praises or criticizes the item.

Text Polarity Detection

Machine learning strategy is the discipline of deciding which changes are most likely to improve a model, based on evidence from errors, data splits, and project constraints. It helps you prioritize the next experiment instead of guessing blindly.

Machine Learning Strategy

Recent progress in deep learning has mainly come from larger data sets and greater computing power.

Learn Before

Related

Learn After