Learn Before
Building an error table for a model evaluated on two image streams.
Case context: A team trains a bird-species classifier on clear laboratory photos and plans to deploy it on images taken by hikers' phones in dim forests. They want to use an error table to understand how the model behaves across the two image streams.
Question: Using the error-table approach, how should the team organize the table, and which axes and row labels should they include to learn whether the model behaves differently on the two streams?
Sample answer: Put the two image streams on the x-axis: laboratory photos and forest phone photos. Put three rows on the y-axis: expert-level baseline error, the model's error on examples from the same source used for training, and the model's error on examples from the deployment source. This layout lets the team compare the model across the two streams and see where performance changes.
Key points:
- x-axis: training-like source and deployment source.
- y-axis: expert baseline error, model error on training-like data, and model error on deployment data.
Rubric: The answer must correctly identify the two columns as the two data sources and the three rows as the three error categories, and explain that the table helps compare behavior across sources.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
In a comparison table for a speech recognizer, what belongs on the x-axis?
In the error table, the horizontal axis shows three different data distributions.
The diagnostic table compares two data _____ across the top and lists three error categories down the side.
Match each table axis or row label to its role in comparing errors across two data sets.
Order the steps for building a comparison table for two data groups.
What is one advantage of adding more entries to a diagnostic error table?
A table can make it easier to compare different kinds of errors across training and development sets.
One row in an error table reports performance on examples the model has _____.
Match each table comparison to the diagnostic insight it provides.
Steps for Interpreting an Error Table Across Two Data Sources
Interpreting a Performance Error Table
Building an error table for a model evaluated on two image streams.
Name the three error rows in the error table