Question: In a concise analytical response, explain the source's claim that an algorithm must perform well on the training set before strong dev/test performance can be expected. Use the 85% and 95% example.

Sample answer: Training-set performance measures the algorithm on examples it has already seen, while dev/test performance concerns examples it has not seen. The source says dev/test performance is usually worse than training performance. Therefore, if the algorithm achieves only 85% accuracy on seen examples, a claim of 95% accuracy on unseen examples is not credible. The algorithm must first improve its training-set performance before strong dev/test results can reasonably be expected.

Key points:

• Training examples have been seen by the algorithm.
• Dev/test examples have not been seen by the algorithm.
• Dev/test performance is usually worse than training-set performance.
• An 85% training result cannot support a 95% result on unseen examples.
• Good training performance must precede expected good dev/test performance.

Rubric: A strong response identifies seen versus unseen examples, states that dev/test performance is usually worse, correctly interprets the 85% training and 95% dev/test claim, and concludes that training performance must improve first.