Learn Before
Diagnose search suboptimality in a scored speech recognition system using approximate search.
Case context: You build a speech recognition system that inputs audio clip A and computes ScoreA(S) = P(S|A). Because exhaustively enumerating all (50,000)^N sentences of length N is impossible, you implement an approximate search algorithm to find the arg max of ScoreA(S). During an evaluation, the system outputs transcription S_approx. However, a manual check reveals that a different English transcription, S_better, has a higher computed score (ScoreA(S_better) > ScoreA(S_approx)).
Question: Diagnose why the system returned S_approx instead of S_better. What does this reveal about the approximate search algorithm's guarantees?
Sample answer: The system returned S_approx instead of S_better because the approximate search algorithm failed to find the sentence that maximizes ScoreA(S). This reveals that the approximate search algorithm is not guaranteed to find the true score-maximizing output, even when a higher-scoring output exists in the search space.
Key points:
- The approximate search algorithm failed to find the score-maximizing transcription.
- Approximate search algorithms are not guaranteed to find the output that maximizes ScoreA(S).
- The higher-scoring transcription S_better was missed due to search approximation rather than scoring error.
Rubric: The response must explain that: 1. The approximate search algorithm failed to identify the optimal output S_better despite it having a higher score. 2. This behavior is due to the inherent limitation of approximate search algorithms, which do not guarantee finding the absolute maximum of the score function.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Data Science
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Machine Learning Strategy
Related
Beam Search as Approximate Inference Search
Why can't a scored inference system exhaustively enumerate all possible output sentences to find the highest-scoring one?
True or False: Approximate search algorithms used in scored inference are guaranteed to find the output that maximizes the score function.
Because exhaustively enumerating all possible outputs is infeasible, a scored inference system must apply a(n) _____ algorithm to find a high-scoring output.
Why must a scored inference system use approximate search rather than exhaustively enumerating all possible output sentences?
Approximate search algorithms used in scored inference are guaranteed to find the output S that maximizes Score_A(S).
In a scored inference system, the formal search objective is to compute the _____ of Score_A(S) over all possible output sentences S.
Match each term to its role in a scored inference system for speech recognition.
Order the steps a scored inference system follows to produce a transcription from an audio clip.
What does the quantity P(S|A) represent when used as Score_A(S) in a scored speech recognition system?
For a vocabulary of 50,000 words, there are (50,000)^N possible output sentences of exactly length N.
Approximate search algorithms are _____ to find the output that maximizes the score function, yet they make search computationally tractable.
Match each challenge in scored inference to the concept that best characterizes it.
Order the reasoning steps that build the argument for why approximate search is necessary in scored inference.
Explain the computational challenge and theoretical limitation of search in a scored inference system.
Diagnose search suboptimality in a scored speech recognition system using approximate search.
State the trade-off of using approximate search algorithms in scored inference.