Diagnosing performance difference between training and dev sets in a speech recognition system
Case context: You train a speech recognition model that achieves high accuracy on your training data. However, when evaluating the model on your dev set, you find that the model performs poorly, indicating a data mismatch problem.
Question: Based on Andrew Ng's recommendations, what specific analysis should you carry out to begin resolving this data mismatch problem?
Sample answer: To begin resolving this data mismatch problem, you should try to understand which properties of the data differ between the training set and the dev-set distributions.
Key points:
- Recognize that the mismatch arises from differing training and dev-set distributions.
- Propose analyzing the data to understand which specific properties differ between the two sets.
Rubric: The learner must state that the next recommended step is to try to understand what properties of the data differ between the training set and the dev-set distributions.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Related
Error Analysis for Data Mismatch
When a data mismatch problem is identified between training and dev sets, what is the recommended first step?
True or False: When a data mismatch problem is found, Andrew Ng recommends understanding which properties of the data differ between the training and dev set distributions.
When a data mismatch problem is found, a recommended step is to understand what _____ of the data differ between the training and dev set distributions.
According to Ng, what is the recommended first step after identifying a data mismatch problem?
When addressing data mismatch, Ng recommends analyzing which properties of the data differ between the training and dev distributions.
When a data mismatch problem is found, the recommended step is to understand which _____ of the data differ between training and dev-set distributions.
Match each term to its role in Ng's data mismatch framework.
Arrange the steps for diagnosing and beginning to address a data mismatch problem in the correct order.
What is the purpose of comparing data properties between training and dev sets when addressing a mismatch problem?
A data mismatch problem is indicated when a model performs poorly on both the training set and the dev set.
Data mismatch is identified when a model performs well on _____ data but poorly on the dev set.
Match each observation or action to its correct interpretation in Ng's data mismatch analysis process.
Arrange these reasoning steps in the correct order for understanding why data properties must be compared when mismatch is found.
Methodology for analyzing distribution discrepancies when data mismatch is detected
Diagnosing performance difference between training and dev sets in a speech recognition system
Analyzing distribution differences to address poor dev set performance