Most textbooks represent vectors (columns) of a tabular dataset $X$ as:
$$X_j=\begin{bmatrix}
    x_{1j} \\
    x_{2j} \\
    x_{3j} \\
    \vdots \\
    x_{nj} \\
\end{bmatrix}$$
So, the dataset can be represented as $$X=\begin{bmatrix}
    X_{1} & X_{2} & X_{3} & \dots  & X_{p}
\end{bmatrix}$$.

University of Michigan - Ann Arbor

A datapoint, instance, sample, or observation, is a unit of data in a dataset. When datasets are tabular, $$x_{ij}$$ usually represents the value of the $$j$$th variable (attribute) in the $$i$$th data point, where $$i=1,2,...,n$$ indicates the index of the data point and $$j=1,2,...,p$$ identifies the variable. This way, a dataset can be represented as an $$n\times p$$ matrix $$X$$, where the row (or vector) $$x_i$$ represents a datapoint that consists of the $$p$$ values of $$x_{i1}, x_{i2}, x_{i3}, \dots , x_{ip}$$ corresponding to each variable. $$X=\begin{bmatrix} x_{11} & x_{12} & x_{13} & \dots & x_{1p} \\ x_{21} & x_{22} & x_{23} & \dots & x_{2p} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ x_{n1} & x_{n2} & x_{n3} & \dots & x_{np} \end{bmatrix} =\begin{bmatrix} x_1^T \\ x_2^T \\ \vdots \\ x_n^T \\ \end{bmatrix}$$

Datapoint

The purpose of a statistical learning model is to use a statistical model and the training data to find the function $\hat{f}$ as an estimate of the unknown function $f$ that maps the matrix of input variables $X$ to the outcome vector $Y$, as best as possible.
$$Y = f(X) + \epsilon \approx \hat{f}(X)$$ 

Purpose of supervised statistical learning

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Formulating the dataset in supervised statistical learning

Data points most fall into one of two categories:
- Qualitative
- Quantitative

Types of Data Points

Most statistics and data science textbooks use the letter $$n$$ to represent the number of data points (instances), and sometimes, they call it the number of observations. The number of distinct data points depends on how the unit of data is defined in the dataset. In tabular datasets, this number corresponds to the number of rows.

Number of data points (instances)

A supervised statistical learning model assumes: 1. A relationship between the matrix of input variables $$X$$ and the outcome $$Y$$. 2. Randomness and independence of $$\epsilon$$ from $$X$$. 3. $$\epsilon$$ should have a mean of 0.

Supervised statistical learning assumptions

After choosing a functional form for $\hat{f}$, based on our domain knowledge or other techniques, we need to estimate the parameters of the model based on the training data. The process of finding optimal values for the parameters of $\hat{f}$ based on the training data is called "training" or "fitting" the model.

Training or Fitting a supervised statistical learning model

In supervised learning problems, a label, also known as a target, is a special attribute of an example that a model aims to predict. It represents the known outcome or ground truth that is not part of the model's input features.

Machine Learning Label

In the supervised statistical learning model $$Y = f(X) + \epsilon$$, the term $$\epsilon$$ is an irreducible error that captures all variation in $$Y$$ that cannot be explained by $$f(X)$$. This includes unmeasurable factors, inherent randomness, and any other sources of variation in $$Y$$ that are independent of $$X$$.

The Error Term $$\epsilon$$ in Supervised Statistical Learning

A prediction problem in statistical learning occurs when there are a significant number of observations and their corresponding outcomes, and the goal is to predict outcomes for a new set of observations where they are unavailable. Let $$X$$ be the set of all input variables and $$Y$$ be the set of corresponding outcomes. If $$\hat{f}$$ is the estimate of the function $$f$$ that maps $$X$$ to $$Y$$, and $$\hat{Y}$$ is the resulting prediction of $$Y$$, assuming the error term averages to zero ($$\bar{\epsilon}=0$$), the following holds:

$$Y=f(X)+\epsilon, \bar{\epsilon}=0 \implies \hat{Y}=\hat{f}(X)$$

Learn Before

Related