Diagnose generalization issues in a speech recognition system.
Case context: Suppose you have developed a speech recognition system that does very well on the training set and on the training dev set. However, it does poorly on your dev set.
Question: Based on this scenario, diagnose the specific problem your speech recognition system is experiencing and explain the underlying reason for this issue.
Sample answer: The speech recognition system is experiencing a data mismatch problem. This occurs because the training set data is a poor match for the dev set data, causing the system to perform well on the training and training dev sets (which share the same distribution) but poorly on the dev set.
Key points:
- Diagnose the issue as a data mismatch problem.
- Acknowledge the system performs well on the training set and training dev set but poorly on the dev set.
- Explain that the training set data is a poor match for the dev set data.
Rubric: The student must diagnose the problem as data mismatch and explain that it is caused by the training set data being a poor match for the dev set data.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Example Data Mismatch Error Pattern
Finding Training Data That Better Matches Difficult Dev Examples
Addressing Data Mismatch by Comparing Data Properties
What defines data mismatch in the context of training and dev/test sets?
Data mismatch means the algorithm fails to generalize even to new data drawn from the training distribution.
Data mismatch is named because the training set data is a poor _____ for the dev/test set data.
Match each data mismatch term to its correct description.
Order the steps for diagnosing a data mismatch problem from training through evaluation.
What is the root cause of a data mismatch problem between training and dev/test sets?
A speech recognition system that does well on training and training dev sets but poorly on the dev set has a data mismatch problem.
An algorithm with data mismatch generalizes well to the _____ distribution but not to the dev/test distribution.
Match each performance pattern to its diagnostic interpretation regarding data mismatch.
Order the reasoning chain that leads from observation to a data mismatch conclusion.
Explain how generalization performance differences between training and dev/test distributions indicate data mismatch.
Diagnose generalization issues in a speech recognition system.
Explain the naming origin of the data mismatch problem.