1. Apply process to your dev and test sets to make sure they come from the same distribution
2. examining examples your algorithm got right as well as the ones it got wrong
3. now train and dev/test set data may come from slightly varying distributions from one another

How to correct incorrect dev/test set data

Overfitting occurs when a model performs significantly better on its training data than on validation or holdout data, resulting in a large generalization gap. This indicates that the model is overly complex and has begun to memorize noise rather than learning generalizable patterns. However, severe overfitting is not always inherently detrimental in deep learning; the best predictive models often exhibit this behavior. The primary goal is to drive the overall generalization error lower, meaning the gap is only problematic if it obstructs that goal.

Overfitting

For a given classification model $$f$$ and a finite dataset $$\mathcal{D}$$ containing $$n$$ examples, the empirical error, denoted as $$\epsilon_\mathcal{D}(f)$$, is the fraction of instances where the model's prediction disagrees with the true label. It is defined mathematically using the indicator function $$\mathbf{1}$$ as:
$$\epsilon_\mathcal{D}(f) = \frac{1}{n}\sum_{i=1}^n \mathbf{1}(f(\mathbf{x}^{(i)}) 
eq y^{(i)})$$

Empirical Error of a Classifier

When evaluating multiple classifiers, such as $$f_1, \dots, f_k$$, on the same test dataset, the probability of obtaining a misleading test set performance score for at least one model increases significantly compared to evaluating a single classifier. For a single classifier $$f$$, a practitioner might be highly confident that its empirical test error $$\epsilon_\mathcal{D}(f)$$ is close to its true population error $$\epsilon(f)$$. However, as the number of classifiers $$k$$ grows, the risk of a false discovery compounds, making it difficult to guarantee that the best-performing model did not simply achieve its seemingly low error rate by chance. This phenomenon directly relates to the statistical challenge of multiple hypothesis testing.

Multiple Hypothesis Testing in Model Evaluation

In machine learning, the mathematical guarantees of test set performance rely on the assumption that a classifier is chosen without any prior contact with the test dataset. If a subsequent model $$f_2$$ is designed after the modeler observes the test set performance of a prior model $$f_1$$, information from the test dataset inevitably leaks to the modeler. Because of this leakage, the test dataset can no longer be viewed as being drawn randomly from the underlying population relative to the modeler's choices. This human-in-the-loop bias is known as adaptive overfitting and compromises the validity of subsequent test set evaluations.

Adaptive Overfitting

A test dataset is a separate partition of data that is held out from the learning process and used strictly to evaluate a model's performance. Because performing well on the training data does not guarantee success on unseen data, the test set serves as a critical measure of the model's true ability to generalize.

Claude

University of Michigan - Ann Arbor

Google

- Training set: to train the model
- Validation set (development set): to tune the model parameters
- Test set: to test the performance of the model on unobserved data

Division of dataset in supervised statistical learning

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Dive into Deep Learning

The observations that are used in a method to build a model for estimating the response variable.

Training data

The training and test data are generated by a probability distribution over datasets called the data-generating process. We typically make an assumptions that the examples in each dataset are independent from each other, and that the training set and test set are identically distributed, drawn from the same probability distribution as each other. This assumption enables us to describe the data-generating process with a probability distribution over a single example. 

The same distribution is then used to generate every train example and every test example. We call that shared underlying distribution the data-generating distribution, denoted $p_{data}$. 

This probabilistic framework and the assumption enable us to mathematically study the relationship between training error and test error.

Data-Generating Process and Data-Generating Distribution (in Machine Learning)

This function randomly shuffles the dataset and splits off a certain percentage of the input samples for use as a training set, and then puts the remaining samples into a different variable for use as a test set. So in this example, we're using a 75-25% split of training versus test data. And that's a pretty standard relative split that's used. It's a good rule of thumb to use in deciding what proportion of training versus test might be helpful.
```
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
```
- Unless optional parameters test_size or train_size are passed in, the method will subset the training and test data randomly into 75% train:25% test
- random_state is an optional parameter used for reproducible output across multiple function calls

Train Test Split Function

- Data is divided 80% as training set, 10% as development set, and 10% as test set.
- Test set should be as large as possible, since a small test set may be accidentally unrepresentative.
- A good common practice is to find the smallest test set that gives us enough statistical power to measure a significant difference between models, since we also need the training set to include as much as data.

Common Practices of Train/Test Set Arrangements in NLP

Test Dataset

A validation dataset (or validation set) is a separate partition of data used to assess the quality of a machine learning model's solution. After training a model iteratively on the training data, its predictive performance is evaluated on this distinct validation dataset to measure how well it has learned the underlying patterns. This evaluation is critical for automatically tuning hyperparameters without relying on the training set, ensuring the model generalizes appropriately to unseen data.

Learn Before

Related

Learn After