A team's training set accuracy is low; what should they do to diagnose the issue?
Case context: A team has trained a speech recognition system, but they notice the algorithm is performing poorly even on the training set itself, not just the dev set.
Question: Based on the source, what should this team do to understand why the algorithm is doing poorly on the training set, and what should they produce from this process?
Sample answer: The team should listen to about 100 training examples that the algorithm is performing poorly on and review them manually. From this review, they should count the errors into different categories, similar to how dev set error analysis is done, to identify the major categories of training set errors and prioritize which issues to address.
Key points:
- Listen to ~100 poorly-performing training examples
- Identify major categories of errors
- Count errors in different categories
- Process mirrors dev set error analysis
Rubric: Full credit for identifying the ~100 example review, the counting of errors by category, and the analogy to dev set error analysis.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Speech Recognition Training Error Categories
When a learning algorithm performs poorly on the training set, what is the recommended first step?
True or False: Reviewing poor training examples involves categorizing and counting errors, similar to dev set error analysis.
If a system performs poorly on the training set, consider listening to about _____ examples the algorithm is doing poorly on.
Match each error analysis concept to its correct description in the context of reviewing training examples.
Order the steps for reviewing poor training examples when the system performs poorly on the training set.
Explain why reviewing ~100 poorly-performing training examples helps diagnose training set errors.
A team's training set accuracy is low; what should they do to diagnose the issue?
Briefly describe what to do when a system is not doing well on the training set.
What is the primary goal of listening to poorly-performing training examples?
True or False: The training set error review process described requires reviewing every single training example.