Using Error Analysis to Investigate Dataset Mismatch

What Should You Do First When Training and Development Data Do Not Match?

True or false: If a model performs well on the training set but poorly on the dev set, a useful next step is to compare how the two datasets differ.

What should you compare when diagnosing a data mismatch?

When a training set and a validation set look different, what should you examine first?

When a model performs differently on two datasets from different populations, a useful first step is to identify which properties of the examples differ between the two sets.

When a train-dev mismatch is suspected, the key diagnostic is to compare which _____ differ across the two datasets.

Match each data-split term to its meaning in diagnosing distribution shift.

Put the steps for identifying a training-development mismatch in order.

Why compare training-set properties with dev-set properties when a model shows a mismatch problem?

True or False: A model that scores well on training data but much worse on dev data is probably underfitting.

A gap between _____ performance and dev-set performance can signal data mismatch.

Match each observation or action to its correct interpretation in a data mismatch diagnosis.

Order the reasoning for diagnosing a training-dev mismatch

How to investigate a dataset mismatch after a dev-set drop

Investigating a gap between training and validation performance in a restaurant review classifier

Comparing dataset characteristics after validation failure