Learn Before
Why separate scoring from search in an AI system?
Question: Explain why it is useful to build an AI system in two parts: first, a model that assigns a score to each possible output; second, a search procedure that chooses the output with the highest score. Describe how this structure helps determine whether a mistake comes from the scoring model or from the search procedure.
Sample answer: This design makes debugging much clearer because the two main sources of error are separated. The scoring model estimates how good each candidate output is for a given input, while the search procedure tries to find the candidate with the largest score. If a clearly correct output receives a higher score than the output chosen by the system, then the search procedure failed to find the best option. If the chosen output scores at least as well as the correct one, then the scoring model is likely the weak part. For example, in a package-delivery system, this can reveal whether the issue is the route-ranking model or the route-selection algorithm.
Key points:
- The scoring model evaluates candidate outputs for a given input
- The search procedure selects the output with the highest estimated score
- Separating the two parts makes debugging easier
- A comparison between a known-good output and the chosen output can diagnose the error source
- If the known-good output scores higher, search is the problem
- If it does not, the scoring model is the likely problem
Rubric: Full credit for explaining that the system is split into a scoring component and a search component, that this split supports a diagnostic comparison between a correct output and the system's output, and that the comparison identifies whether the scoring model or the search procedure is responsible for the error.
0
1
Tags
Data Science
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Machine Learning Yearning @ DeepLearning.AI
Related
Why Scored Decoding Often Uses Approximate Search
Distance-Based Scoring Function
Semantic Matching Scoring Function
In a speech-to-text model, what does Score_audio(phrase) represent?
True or False: Assigning scores to candidates does not eliminate the need for a separate step that finds the highest-scoring choice.
Ranking by Score
Match each scoring-function element to its role in a route-planning example.
Put the score-and-search workflow in order.
Why separate scoring from search in an AI system?
Use the score comparison to identify the source of the error.
What are the two parts of a typical scoring-and-search design?
When is the optimization-check idea applicable?
True or False: A common machine learning pipeline uses an exact scoring model followed by an exact maximization procedure.