Learn Before
Diagnose a speech recognition error using the scoring function framework.
Case context: A speech recognition system computes Score_A(S) = P(S|A) for an audio clip A. For a particular clip, the system outputs the sentence 'I love your city' but the correct transcription is 'I love acity.' The engineer checks Score_A('I love acity') and finds it is lower than Score_A('I love your city'), even though 'I love acity' is the correct answer.
Question: Based on this scoring comparison, should the engineer conclude the problem is in the scoring function or in the search/maximization algorithm? Explain your reasoning.
Sample answer: Since the correct sentence 'I love acity' received a lower score than the incorrect output 'I love your city', the scoring function itself is failing to rank the correct answer highest. This means the problem lies in the scoring function (it needs to be improved, e.g., via better training data or model), not in the search algorithm, because even a perfect search would have found and returned the higher-scoring but wrong sentence.
Key points:
- Compare Score_A of the correct sentence to Score_A of the algorithm's output
- If the correct sentence scores lower, the scoring function is at fault
- If the correct sentence scores higher or equal but wasn't chosen, the search algorithm is at fault
- This reasoning reflects the Optimization Verification test
Rubric: Full credit for correctly identifying that a lower score for the correct sentence indicates a scoring function problem, and for explaining that this conclusion follows from the Optimization Verification test logic described in the source.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Data Science
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Machine Learning Yearning @ DeepLearning.AI
Related
Approximate Search in Scored Inference
Distance-Based Scoring Function
Semantic Matching Scoring Function
In Andrew Ng's speech recognition example, what does Score_A(S) represent?
True or False: A scoring function alone is sufficient; no separate search or maximization step is needed.
An inference system searches for the output that _____ the scoring function.
Match each scoring function component to its role in the speech recognition example.
Order the steps of the scoring function design pattern in AI system design.
Explain why learning a scoring function and searching separately is a useful AI design pattern.
Diagnose a speech recognition error using the scoring function framework.
What two components make up the common AI design pattern described for scoring functions?
When can you apply the Optimization Verification test according to the source?
True or False: In the common design pattern, both the scoring function and the maximization algorithm are typically exact rather than approximate.