Concept

False Discovery in Model Evaluation

In machine learning, a false discovery occurs when a classifier's empirical error on a test set appears exceptionally low due to statistical noise or random chance, rather than representing true predictive ability on the underlying population. While the probability of a misleading evaluation might be small (e.g., 5%5\%) when evaluating a single model, this risk compounds when a test set is reused to evaluate multiple classifiers simultaneously, drastically increasing the likelihood that at least one model will receive an unjustifiably favorable score.

0

1

Updated 2026-05-03

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L