The Recall of a cat classifier is the percentage of all cat images in the dev or test set that it correctly labeled as a cat.

Google

During development, a team will try many ideas about algorithm architecture, model parameters, and choice of features. Having a single-number evaluation metric such as accuracy allows the team to sort models according to their performance on the metric and quickly decide what is working best. It also speeds up decisions when selecting among a large number of classifiers by giving a clear preference ranking and a clear direction for progress.

Single-Number Evaluation Metric

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

The Precision of a cat classifier is the fraction of images in the dev or test set that it labeled as cats that really are cats.

Precision (Cat Classifier)

Recall (Cat Classifier)

If both Precision and Recall matter, one standard way to combine them into a single number is to take the average of precision and recall.

Combining Precision and Recall into a Single Number

If the accuracy of a cat classifier is tracked separately in four key markets, this gives four metrics. By taking an average or weighted average of these four numbers, one ends up with a single number metric. Taking an average or weighted average is one of the most common ways to combine multiple metrics into one.

Combining Multiple Metrics via Averaging

Without a specific dev set and metric, a team may have to incorporate a new classifier into an app and manually try the app to judge whether the classifier is an improvement. Having a dev set and metric allows the team to quickly detect which ideas are giving small or large improvements, and therefore quickly decide which ideas to keep refining and which ones to discard.

Learn Before

Related