- Future unseen data (test set) will have the same properties as the current training set.
- Thus, models that are accurate on the training set are expected to be accurate on the test set.
- But that may not happen if the trained model is tuned too specifically to the training set.

University of Michigan - Ann Arbor

Generalizability refers to an algorithm's ability to give accurate predictions for new, previously unseen data.

Generalizability of a supervised statistical model

Assumptions of Generalizability of a supervised statistical model

In machine learning, generalization refers to the formidable challenge of finding a model whose parameters lead to accurate predictions on previously unseen data. While deep learning optimization algorithms can often easily find parameters that minimize the loss on the training set, ensuring that those learned parameters adapt well and maintain high predictive accuracy on new, independent datasets is the ultimate objective of the learning process.

Generalization

Use this plot, which shows accuracy scores for different values of a regularization parameter lambda, to answer.

In this example, what value of lambda is the best choice for generalization?

According to the 'no free lunch' theorem (Wolpert and Macready, 1995), there is no universally superior learning algorithm. Any machine learning algorithm that generalizes well on data with certain distributions will inherently perform worse on other distributions. Consequently, given a finite training set, achieving high performance requires a model to incorporate specific assumptions or inductive biases that are well-suited to the underlying distribution of the target task.

Learn Before

Related