Based on the supervised learning scenario described below, identify the outcome variable (also known as the target or dependent variable) and briefly explain why it is classified as such.

Identifying the Outcome Variable in a Predictive Model

A data scientist is developing a supervised learning model to determine whether an email is 'spam' or 'not spam'. The model is trained on a dataset of thousands of emails, each with a corresponding label. In this scenario, what does the outcome vector, denoted as Y, consist of?

A real estate company is building a statistical model to predict the selling price of a house based on its features, such as square footage, number of bedrooms, and location. In the context of a supervised learning problem, what specific quantity represents the outcome value (denoted as $y_i$) for a single house in their dataset?

Identifying the Outcome in a Regression Problem

Label noise occurs when a training dataset contains incorrectly or randomly assigned labels, introducing intrinsic variability into the learning process. For example, when predicting mortality among patients, the true outcome may be ambiguous or recorded incorrectly. While neural networks can eventually interpolate these noisy labels, their presence makes regularization techniques like early stopping crucial to prevent overfitting.

Definition of Label Noise

In supervised learning problems, a label, also known as a target, is a special attribute of an example that a model aims to predict. It represents the known outcome or ground truth that is not part of the model's input features.

University of Michigan - Ann Arbor

Google

Claude

The purpose of a statistical learning model is to use a statistical model and the training data to find the function $\hat{f}$ as an estimate of the unknown function $f$ that maps the matrix of input variables $X$ to the outcome vector $Y$, as best as possible.
$$Y = f(X) + \epsilon \approx \hat{f}(X)$$ 

Purpose of supervised statistical learning

In a machine learning dataset, a feature (also referred to as an input or covariate) is an individual, measurable attribute of a given data point. A model relies on these numerical representations to make its predictions.

Machine Learning Feature

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Reference of Foundations of Large Language Models Course

Dive into Deep Learning

A supervised statistical learning model assumes:
1. A relationship between the matrix of input variables $X$ and the outcome $Y$.
2. Randomness and independence of $\epsilon$ from $X$.
3. $\epsilon$ should have a mean of 0.

Supervised statistical learning assumptions

Most textbooks represent vectors (columns) of a tabular dataset $X$ as:
$$X_j=\begin{bmatrix}
    x_{1j} \\
    x_{2j} \\
    x_{3j} \\
    \vdots \\
    x_{nj} \\
\end{bmatrix}$$
So, the dataset can be represented as $$X=\begin{bmatrix}
    X_{1} & X_{2} & X_{3} & \dots  & X_{p}
\end{bmatrix}$$.

Formulating the dataset in supervised statistical learning

The error term $\epsilon$ indicates the residual of fitted values compared to the observed outcomes, i.e., $\epsilon$ accounts for variation in $Y$ that cannot be explained by the estimated function $f$:
$$Y=f(X)+\epsilon$$ 

$\epsilon$ in supervised statistical learning

After choosing a functional form for $\hat{f}$, based on our domain knowledge or other techniques, we need to estimate the parameters of the model based on the training data. The process of finding optimal values for the parameters of $\hat{f}$ based on the training data is called "training" or "fitting" the model.

Training or Fitting a supervised statistical learning model

Machine Learning Label

In machine learning, image data is often represented numerically. For instance, a color photograph can be represented as three grids of numerical values corresponding to the brightness of red, green, and blue light at each pixel location. Consequently, a $$200 \times 200$$ pixel color image consists of $$200 \times 200 \times 3 = 120000$$ numerical features.

Image Data Representation Example

Electronic health records provide a classic example of tabular data used in machine learning. For example, when predicting whether a patient will survive the next 30 days, the features might include a set of readily available attributes such as age, vital signs, comorbidities, current medications, and recent procedures. The label used for training the model would be a binary value indicating if the patient survived within that 30-day period.

Electronic Health Record Data Example

One of the most significant advantages of deep learning is its ability to replace the labor-intensive process of manual feature engineering. In traditional machine learning, domain experts crafted specific algorithms to transform raw perceptual data into feature vectors. Deep learning replaces these domain-specific preprocessing steps with automatically tuned filters that are learned jointly from the data, yielding superior accuracy and providing a unified approach across diverse fields.

Automated Feature Engineering

When machine learning inputs are structured as fixed-length vectors, the constant length of these vectors—meaning the exact number of numerical features—is defined as the dimensionality of the data. For instance, a $$200 	imes 200$$ pixel color image represented by red, green, and blue values has a dimensionality of $$200 	imes 200 	imes 3 = 120000$$.

Learn Before

Related

Learn After