Essay

Why does establishing dev/test sets quickly benefit a team working on a new application?

Question: Explain why Andrew Ng recommends establishing dev/test sets and a metric in less than a week for a brand new application, and why this same urgency does not apply to mature applications.

Sample answer: Establishing dev/test sets quickly gives the team a well-defined, shared target to aim for from the start, preventing wasted time and misdirected effort. It is better to get going with something imperfect than to overthink the initial setup, since early progress and iteration matter more than perfection at this stage. This urgency does not apply to mature applications because those teams already have a working system and established direction; they can afford to spend more time, even months, refining their dev/test sets to gain more precise, higher-quality evaluation criteria.

Key points:

  • Quick dev/test sets give the team a well-defined target
  • Imperfect but fast is better than overthinking
  • One-week timeline applies specifically to new applications
  • Mature applications may spend months acquiring better dev/test sets
  • Mature teams already have direction so speed is less critical

Rubric: Full credit explains both the benefit of speed for new applications (well-defined target, avoiding overthinking) and the contrast with mature applications (already have direction, can afford to spend more time). Partial credit addresses only one side.

0

1

Updated 2026-07-10

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI