Single-Score Model Evaluation

An Early Dev/Test Setup Speeds Model Iteration

Revise the evaluation setup when it stops matching the goal

Error Analysis

Reviewing Both Missed and Correct Dev-Set Examples

After a development set and a test set have been chosen, what are they mainly used for?

True or False: A development set score can provide a quick check on whether a new idea is moving a model in the right direction.

Dev and test sets give a quick check on how well a team’s _____ is performing.

Match each dev/test-set action with its main effect.

Arrange the basic workflow for using development and test sets to improve a model.

Explain why a dev set, test set, and metric help a machine learning team work efficiently.

Diagnosing a project with no evaluation set

In one to three sentences, explain how a validation set can guide which model ideas deserve more work.

How does dev set performance help a team choose which ideas to keep improving?

What do teams usually try once dev and test sets are fixed?

What do teams typically do after they have set up dev and test sets?

True or False: With a fixed development set and a single evaluation metric, a team can quickly tell whether a new idea is helping by a little or by a lot.

A development set and a chosen evaluation metric help a team compare ideas quickly and see whether each change is making progress.