Recognizing Distribution Mismatch from Error Measurements

Collect Training Examples That Resemble the Hard Development Cases

Compare training and development data properties after a mismatch is found

What best describes a mismatch between training data and evaluation data?

A data mismatch problem means the model performs poorly on both training-like data and the development/test data.

Data mismatch and distribution fit

Match each data-distribution term with its description.

Put the diagnostic workflow for dataset shift in the correct order.

What most directly causes a mismatch between training data and evaluation data?

A voice-to-text system can perform well on the training set and training-dev set but still do poorly on the dev set if the dev data comes from a different distribution.

Training Distribution and Evaluation Distribution

Match each performance pattern to the most likely diagnosis of the model's problem.

Put the steps in order for diagnosing a distribution mismatch from model performance.

Explain what a performance gap across two data distributions suggests

Explain why a voice-command model looks strong on one split but weak on another.

Why is this called data mismatch?