Kitten Uploads Revealing Dev/Test Distribution Mismatch

When a team discovers the dev/test set distribution differs from actual deployment distribution, what is the recommended course of action?

True or False: A dev/test set that differs in distribution from actual deployment data still provides a reliable signal for model improvement.

When the dev/test set is not representative of the actual distribution, Ng recommends teams _____ the dev/test sets.

Match each term related to dev/test set distribution with its correct description.

Arrange the steps a team should follow after suspecting their dev/test set no longer matches the actual distribution.

Why does a dev/test set whose distribution mismatches the actual distribution fail to guide a development team effectively?

True or False: According to Machine Learning Yearning, the dev/test sets should reflect the actual distribution the system needs to perform well on.

According to Machine Learning Yearning p. 24: 'The actual distribution you need to do well on is _____ from the dev/test sets.'

Match each scenario to the correct conclusion about whether the dev/test set should be updated.

Arrange the logical reasoning steps that explain why a non-representative dev/test set must be updated.

Analyze the consequences of a mismatch between dev/test distribution and actual distribution

Diagnose a scenario where actual user uploads differ from the dev/test set

State the required action when a dev/test set distribution is not representative