Case Study

Choose a weighted metric for regional classifier evaluation.

Case context: A team is building a spam filter and tracking accuracy separately for four deployment areas: North America, South America, Europe, and Asia. The Asia segment has far more daily messages than North America, while North America produces the largest revenue per active account. The team wants one score to compare model versions.

Question: Should the team summarize these four accuracies with an equal average or a weighted average? Explain how the final score should be built so it matches the company's priorities.

Sample answer: A weighted average is the better choice. Giving each region the same influence would ignore major differences in traffic volume and business value, which could lead the team to pick a model that looks good on paper but is weak where it matters most. The team should assign a weight to each region's accuracy, such as a weight based on message volume, account value, or another strategic measure, and then combine the four values into one weighted formula for model selection.

Key points:

  • Select a weighted average rather than an equal average.
  • State that equal averaging overlooks differences in user volume and business impact.
  • Define weights using factors such as traffic, revenue, or strategic importance.
  • Combine all four regional accuracies into one optimization target.

Rubric: The response must choose a weighted average, explain why equal averaging is not appropriate when the regions differ, and describe assigning weights using business or usage priorities.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI