Learn Before
Contrast the target actions required when debugging an inference system's search algorithm versus its scoring function.
Question: Analyze the two primary failure modes of an inference system: a search algorithm problem and a scoring function (objective) problem. Contrast how the target actions you must take to improve the system differ depending on which of these two issues is identified as the root cause of a wrong output.
Sample answer: When debugging an inference system that yields incorrect output, you must distinguish between a search algorithm failure and a scoring function failure. If the root cause is a search algorithm problem (where the approximate search failed to find the value of S that maximizes the score), the developer must prioritize improving the search algorithm itself. Conversely, if the root cause is a scoring function problem (where the estimated scores themselves are inaccurate), the developer must focus on improving the learning algorithm that estimates the scores rather than modifying the search algorithm.
Key points:
- A search algorithm problem means the approximate search failed to find the value that maximizes the score.
- A scoring function problem means the estimates for the scores were inaccurate.
- If the search algorithm failed, priority must go to improving the search algorithm.
- If the scoring function failed, priority must go to improving the learning algorithm that estimates the score.
Rubric: The answer must accurately define the actions for both failure modes. It should clearly state that a search algorithm problem requires working on the search algorithm itself, while a scoring function (objective) problem requires working on the learning algorithm that estimates the score.
0
1
References
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Machine Learning Yearning (Deeplearning.ai)
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Search Algorithm Problem in Inference
Scoring Function Problem in Inference
Optimization Verification Test
When an inference system returns a wrong output, which two root causes should a developer distinguish between to debug effectively?
Identifying whether the search algorithm or scoring function caused an inference failure determines which component you should prioritize improving.
If the approximate search algorithm fails to find the value of S that maximizes Score_A(S), the correct fix is to work on improving the _____ algorithm.
Match each inference failure type to its precise description as defined in Machine Learning Yearning.
Arrange the steps of the inference debugging process described in MLY in the correct logical order.
According to MLY, what is a 'scoring function problem' in the context of debugging inference systems?
Randomly choosing whether to improve the search algorithm or scoring function is an acceptable debugging strategy when an inference system produces wrong outputs.
When the scoring function is identified as the inference failure cause, you should work on the _____ algorithm that estimates Score_A(S).
Match each identified inference failure cause to the correct recommended action from Machine Learning Yearning.
Arrange the reasoning steps used to determine whether an inference failure is caused by an inaccurate scoring function.
Contrast the target actions required when debugging an inference system's search algorithm versus its scoring function.
Diagnosing incorrect transcriptions in a speech recognition inference system
Consequence of randomly choosing between improving search or scoring during inference debugging