Which comparison most clearly indicates a data mismatch problem in the case where training error is 4%, same-distribution unseen error is 5%, and dev error is 14%?

A train error of 0.8%, a same-source holdout error of 1.0%, and a development error of 7.5% mainly show overfitting.

In the distribution-shift example, the classifier has _____ error on the dev set.

Match each error metric in the distribution-shift example to its value.

Order the steps for diagnosing a distribution-shift problem in model evaluation.

What does a 0.4% gap between training error (2.0%) and error on other data drawn from the same distribution (2.4%) suggest?

If training error is 2%, human-level error is 1%, and development error on a different data source is 12%, the 1% gap between training error and human-level error is the biggest issue to fix.

For unseen data drawn from the same distribution as the training set, the error is _____.

Match each comparison to the kind of error it measures.

Order the evidence showing that distribution shift is the main issue in a speech-command classifier.

Explain how two error comparisons reveal a distribution mismatch.

Diagnosing a Dev-Set Error Spike in a Product Review Classifier

Meaning of error on the training-like distribution