Question: In a concise analytical response, explain why training performance alone or dev/test performance alone is insufficient to express the supplied informal definition of variance.

Sample answer: Informal variance is defined as how much worse an algorithm does on the dev or test set than on the training set. Therefore, it depends on a comparison between training performance and dev or test performance. Either result alone omits the gap that the term variance informally describes.

Key points:

• Training performance is the reference.
• Dev or test performance is compared with training performance.
• Variance is the amount by which dev or test performance is worse.
• A single performance result does not express the comparison.

Rubric: A strong response accurately identifies both sides of the comparison, explains that variance is the amount of worsening between them, and states why either performance result alone is insufficient.