Learn Before
Concept
False Discovery in Model Evaluation
In machine learning, a false discovery occurs when a classifier's empirical error on a test set appears exceptionally low due to statistical noise or random chance, rather than representing true predictive ability on the underlying population. While the probability of a misleading evaluation might be small (e.g., ) when evaluating a single model, this risk compounds when a test set is reused to evaluate multiple classifiers simultaneously, drastically increasing the likelihood that at least one model will receive an unjustifiably favorable score.
0
1
Updated 2026-05-03
Tags
D2L
Dive into Deep Learning @ D2L