Learn Before
Explain the computational challenge and theoretical limitation of search in a scored inference system.
Question: In a scored inference system, such as a speech recognition system that scores possible output sentences S given an audio input A, we must find the sentence S that maximizes ScoreA(S). Explain why exhaustive enumeration of all possible sentences is computationally impractical, and analyze the consequence of using an approximate search algorithm instead.
Sample answer: Exhaustive enumeration is computationally impractical because the number of possible output sentences grows exponentially with sentence length. For example, with a vocabulary of 50,000 words, there are (50,000)^N possible sentences of length N, which is far too large to evaluate. To make this tractable, we must use an approximate search algorithm to search for the output S that optimizes ScoreA(S). The major limitation of this approach is that approximate search algorithms are not guaranteed to find the actual value of S that maximizes the score function.
Key points:
- Exhaustive search is infeasible due to the exponential growth of the sentence search space (e.g., 50,000^N).
- Approximate search is required to make finding high-scoring outputs computationally tractable.
- Approximate search algorithms do not guarantee finding the output S that maximizes ScoreA(S).
Rubric: The answer should address two parts: 1) the exponential growth of the search space (e.g., vocabulary size raised to the power of N) that makes exhaustive search infeasible, and 2) the fact that approximate search algorithms are necessary for computational tractability but do not guarantee finding the score-maximizing output.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Data Science
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Machine Learning Strategy
Related
Beam Search as Approximate Inference Search
Why can't a scored inference system exhaustively enumerate all possible output sentences to find the highest-scoring one?
True or False: Approximate search algorithms used in scored inference are guaranteed to find the output that maximizes the score function.
Because exhaustively enumerating all possible outputs is infeasible, a scored inference system must apply a(n) _____ algorithm to find a high-scoring output.
Why must a scored inference system use approximate search rather than exhaustively enumerating all possible output sentences?
Approximate search algorithms used in scored inference are guaranteed to find the output S that maximizes Score_A(S).
In a scored inference system, the formal search objective is to compute the _____ of Score_A(S) over all possible output sentences S.
Match each term to its role in a scored inference system for speech recognition.
Order the steps a scored inference system follows to produce a transcription from an audio clip.
What does the quantity P(S|A) represent when used as Score_A(S) in a scored speech recognition system?
For a vocabulary of 50,000 words, there are (50,000)^N possible output sentences of exactly length N.
Approximate search algorithms are _____ to find the output that maximizes the score function, yet they make search computationally tractable.
Match each challenge in scored inference to the concept that best characterizes it.
Order the reasoning steps that build the argument for why approximate search is necessary in scored inference.
Explain the computational challenge and theoretical limitation of search in a scored inference system.
Diagnose search suboptimality in a scored speech recognition system using approximate search.
State the trade-off of using approximate search algorithms in scored inference.