Case context: A team is evaluating two classifiers. Classifier A runs in 80ms and achieves 90% accuracy. Classifier B runs in 60ms and achieves 88% accuracy. The team has set running time as a satisficing metric with an acceptable threshold of at most 100ms.

Question: Using the satisficing metric rule for running time, which classifier should the team select, and why?

Sample answer: Both Classifier A (80ms) and Classifier B (60ms) meet the satisficing threshold of at most 100ms, so running time is no longer a distinguishing factor between them. Since running time is only a satisficing metric and not something to be further optimized, the team should select the classifier with the better performance on the remaining metric, which is accuracy. Classifier A has higher accuracy (90% vs 88%), so the team should choose Classifier A.

Key points:

• Both classifiers meet the 100ms satisficing threshold for running time.
• A satisficing metric does not need to be minimized beyond meeting the threshold.
• Since running time is satisfied for both, the decision should be based on the other metric.
• Classifier A should be chosen because it has higher accuracy (90% vs 88%).

Rubric: Full credit responses recognize that both classifiers satisfy the running time threshold, so running time does not need to be minimized further, and correctly select the classifier with higher accuracy. Partial credit for correctly identifying both classifiers meet the threshold but failing to select the higher-accuracy classifier.