Learn Before
Speech Recognition Debugging
Case context: You are developing a speech recognition system. For a specific audio clip, the correct transcription is "I love machine learning" (), but your system outputs "I love robots" (). Your system uses a scoring function and an approximate search algorithm to find the transcription with the highest score.
Question: How would you use the Optimization Verification test to diagnose why the system output "I love robots"? What specific measurements would you take, and how would you interpret the results to decide whether to fix the search algorithm or the scoring function?
Sample answer: I would compute and . Then, I would check if Score_A("I love machine learning") > Score_A("I love robots"). If this inequality is true, it means the scoring function is working properly (it prefers the correct answer), so I should fix the approximate search algorithm because it failed to find the transcription with the highest score. If the inequality is false, it means the scoring function erroneously assigned a higher (or equal) score to the incorrect transcription, so I should fix the scoring function computation.
Key points:
- Compute and .
- Check whether .
- Blame the search algorithm if the inequality holds.
- Blame the scoring function if the inequality does not hold.
Rubric: A strong answer will explicitly mention computing the scores for the two specific transcriptions provided and correctly explain how to map the inequality to the component at fault.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Applying Optimization Verification Across Dev Set Errors
Approximate Scoring Function Plus Approximate Maximization Pattern
Optimization Verification for Machine Translation
Optimization Verification for Reinforcement Learning Reward Functions
Using a Superior Human Output in Optimization Verification
Diagnosing Errors with Optimization Verification
To apply the Optimization Verification test for a given input , you must know how to compute a _____ that indicates how good a response is to that input.
Optimization Verification Variables
Steps to Perform Optimization Verification
Interpreting the Test Outcomes
Speech Recognition Debugging
Condition for Blaming Optimization
Preconditions for Optimization Verification
Applicability of the Test