Case context: A team is building a classifier that must return a prediction within 120 milliseconds on average. That speed requirement has already been decided. The team now has one other important measure of success: accuracy.

Question: After the time limit has been fixed, how should the team treat accuracy, and why?

Sample answer: The team should make accuracy the optimizing metric and try to maximize it, as long as the classifier still satisfies the 120-millisecond time requirement. The speed requirement acts as a threshold that must be met, while accuracy is the metric to improve among solutions that meet that threshold.

Key points:

• The 120-millisecond limit is a satisficing requirement.
• Accuracy is the remaining metric to optimize.
• Accuracy should be maximized, not just checked against a cutoff.
• The model must still meet the speed requirement while improving accuracy.

Rubric: Full credit requires identifying accuracy as the optimizing metric after the time constraint is fixed, and explaining that accuracy should be maximized subject to meeting the runtime threshold. Partial credit is available for noting that accuracy matters without stating the subject-to relationship.