Question: Analyze how designating N-1 criteria as satisficing metrics and only one as an optimizing metric facilitates model selection compared to trying to maximize all N criteria simultaneously. Draw from the examples and criteria described in the source text.

Sample answer: When a team faces N different criteria (such as binary file size, running time, and accuracy), trying to optimize all of them at once makes it difficult to compare different models. The N-1 satisficing and 1 optimizing framework simplifies this. By setting threshold values for N-1 of the criteria (e.g., binary file size and running time must be under a certain limit), these criteria become binary constraints (pass/fail). Then, the team can focus solely on maximizing or minimizing the single optimizing metric (e.g., accuracy) among the models that meet all satisficing thresholds. This provides a clear, unambiguous rule for selecting the best model.

Key points:

• Setting threshold values for N-1 criteria transforms them into binary constraints.
• Designating a single optimizing metric provides a clear rule for model comparison.
• Helps navigate trade-offs among N criteria like binary file size, running time, and accuracy.

Rubric: To earn full credit, the answer must: 1) Explain how setting threshold values for N-1 criteria transforms them into binary constraints (satisficing metrics); 2) Describe how having a single optimizing metric provides a clear, unambiguous rule for comparing models; 3) Use the criteria of binary file size, running time, and accuracy from the text to illustrate the concept.