Case Study

Building an error table for a model evaluated on two image streams.

Case context: A team trains a bird-species classifier on clear laboratory photos and plans to deploy it on images taken by hikers' phones in dim forests. They want to use an error table to understand how the model behaves across the two image streams.

Question: Using the error-table approach, how should the team organize the table, and which axes and row labels should they include to learn whether the model behaves differently on the two streams?

Sample answer: Put the two image streams on the x-axis: laboratory photos and forest phone photos. Put three rows on the y-axis: expert-level baseline error, the model's error on examples from the same source used for training, and the model's error on examples from the deployment source. This layout lets the team compare the model across the two streams and see where performance changes.

Key points:

  • x-axis: training-like source and deployment source.
  • y-axis: expert baseline error, model error on training-like data, and model error on deployment data.

Rubric: The answer must correctly identify the two columns as the two data sources and the three rows as the three error categories, and explain that the table helps compare behavior across sources.

0

1

Updated 2026-08-12

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI