Short Answer

Why are formal significance checks rarely used during dev-set iteration?

Question: In one to three sentences, explain the view on using significance testing to judge small improvements on a development set.

Sample answer: Teams usually skip formal significance tests when they are comparing iterative dev-set changes. The main reason is that these tests are often not very helpful for everyday model development, although they can still matter in some research-paper settings.

Key points:

  • Routine model iteration usually does not rely on formal tests.
  • Small dev-set gains are often judged by practical impact instead.
  • Research publication is the common exception.

Rubric: The answer should say that these tests are not commonly used for routine dev-set comparison and that they are typically seen as of limited value for day-to-day progress; mentioning the research exception makes the response more complete.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Machine Learning Strategy

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Yearning @ DeepLearning.AI