Learn Before
Averaging Subgroup Scores into a Single Evaluation Metric
When the same performance measure is calculated separately for several regions or customer segments, the resulting subgroup scores can be combined into one evaluation metric. Use a simple average when every subgroup should have equal influence. Use a weighted average when subgroup influence should differ, choosing weights that reflect the evaluation objective, such as relative traffic or business importance. The combined score simplifies comparison and tracking, but its interpretation depends on the chosen weighting.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Precision for a Cat Detector
Recall for a Positive-Class Detector
One Score from Precision and Recall
Using a Dev Set to Compare Model Versions
What is the main advantage of using one evaluation score while developing models?
A single score can help a team rank many models quickly.
Single-number metrics for model selection
Match each development choice to its role in testing model ideas.
Put the model-selection process with one metric in order.
Why a single evaluation score speeds model development
Use one primary score to compare many candidate models.
What two kinds of guidance does a single-number score provide?
What evaluation strategy best helps you choose quickly among many model candidates?
A single evaluation score can help a team choose among competing models and point the team toward the next improvement.
Averaging Subgroup Scores into a Single Evaluation Metric
Learn After
How should separate regional accuracies be combined into one summary metric?
True or False: A simple average of several performance metrics can be used to summarize them with one overall score.
A metric formed by averaging accuracy across four regional markets is a _____ metric.
What does averaging accuracy scores from four regions give you?
True or False: A simple average or a weighted average is a common way to combine several metrics into one score.
Evaluating a recommendation model separately in four customer segments gives you _____ metrics before you combine them.
Match each model-combination idea to its description.
Order the steps for combining four regional conversion rates into one decision metric.
Why choose a weighted average instead of an ordinary average when combining several regional accuracy scores?
Each region in a four-region performance summary can contribute one accuracy score.
Combining several metrics with an average or weighted average is one of the most _____ ways to produce a single score.
Match each idea to its role in combining several site scores into one evaluation number.
Put the steps for deciding on one combined score in order.
Why use a weighted average instead of a plain average when combining accuracy scores from several customer segments?
Choose a weighted metric for regional classifier evaluation.
Why is it useful to collapse several region-level scores into one metric?