Single-Number Evaluation Metric

Initial Dev/Test Set and Metric Helps Quick Iteration

Changing Dev/Test Sets or the Metric When They No Longer Guide the Team

Error Analysis

Double-Checking Labels of Both Misclassified and Correctly Classified Dev Examples

What is the primary purpose of the dev and test sets once they are defined?

True or False: Measuring an idea's performance on the dev set lets a team quickly decide if it is heading in the right direction.

The dev and test sets allow a team to quickly see how well their _____ is doing.

Match each action related to dev/test sets with its described effect.

Order the steps of using dev/test sets to guide iteration on a machine learning idea.

Explain why having dev/test sets and a metric is important for a team's workflow.

A team tries several algorithm changes without a dev set. Diagnose the resulting problem.

In one to three sentences, describe how the dev set helps a team decide which ideas to keep refining.

In one to three sentences, describe how the dev set helps a team decide which ideas to keep refining.

What kind of things might a team try after defining a dev set and test set?

What kind of things might a team try after defining a dev set and test set?

True or False: Having a dev set and metric allows a team to quickly detect which ideas give small or large improvements.

True or False: Having a dev set and metric allows a team to quickly detect which ideas give small or large improvements.