Google

A scoring function assigns a numeric value to each candidate output for a given input. An inference procedure can then look for the candidate with the highest score. A common design is to learn an approximate scoring rule and then use an approximate search method to find a strong output.

Scoring Function

A system can assign a score to each possible output and still be unable to search every option exactly, because the space of candidates may be enormous. In that situation, the system uses an approximate search method to find a strong candidate quickly. Such methods are practical, but they do not guarantee finding the absolute highest-scoring output.

Why Scored Decoding Often Uses Approximate Search

In a distance-based scoring function for knowledge graphs, the plausibility of a fact is measured by the distance between entities. This relationship is typically expressed as $$h + r \sim t$$, where $$h$$ represents the head entity, $$r$$ represents the relation, and $$t$$ represents the tail entity.

Distance-Based Scoring Function

In a similarity-based (or semantic matching) scoring function for knowledge graphs, the plausibility of a fact is measured by matching the latent semantics of entities and relations. This relationship is typically expressed using a bilinear formulation, such as $$h^T M_r \sim t^T$$, where $$h$$ and $$t$$ represent the head and tail entity vectors, and $$M_r$$ represents the relation matrix.

Semantic Matching Scoring Function

In a speech-to-text model, what does Score_audio(phrase) represent?

True or False: Assigning scores to candidates does not eliminate the need for a separate step that finds the highest-scoring choice.

A candidate selector chooses the option that _____ the evaluation score for the current input.

Ranking by Score

Match each scoring-function element to its role in a route-planning example.

Put the score-and-search workflow in order.

Question:
Explain why it is useful to build an AI system in two parts: first, a model that assigns a score to each possible output; second, a search procedure that chooses the output with the highest score. Describe how this structure helps determine whether a mistake comes from the scoring model or from the search procedure.

Sample answer:
This design makes debugging much clearer because the two main sources of error are separated. The scoring model estimates how good each candidate output is for a given input, while the search procedure tries to find the candidate with the largest score. If a clearly correct output receives a higher score than the output chosen by the system, then the search procedure failed to find the best option. If the chosen output scores at least as well as the correct one, then the scoring model is likely the weak part. For example, in a package-delivery system, this can reveal whether the issue is the route-ranking model or the route-selection algorithm.

Key points:
- The scoring model evaluates candidate outputs for a given input
- The search procedure selects the output with the highest estimated score
- Separating the two parts makes debugging easier
- A comparison between a known-good output and the chosen output can diagnose the error source
- If the known-good output scores higher, search is the problem
- If it does not, the scoring model is the likely problem

Rubric:
Full credit for explaining that the system is split into a scoring component and a search component, that this split supports a diagnostic comparison between a correct output and the system's output, and that the comparison identifies whether the scoring model or the search procedure is responsible for the error.

Why separate scoring from search in an AI system?

Case context:
A voice-to-text system assigns a score to each candidate transcript for a 12-second recording using Score_R(T) = P(T|R). For one recording, the system outputs “Please send the report tomorrow,” but the correct transcription is “Please send the support tomorrow.” The engineer checks the scores and finds that Score_R("Please send the support tomorrow") is lower than Score_R("Please send the report tomorrow").

Question:
Should the engineer focus on the scoring function or on the search/maximization procedure? Explain why.

Sample answer:
The scoring function is the likely problem. The correct transcript has a lower score than the transcript that was selected, so the score assignments are not ranking the true answer above the incorrect one. If the search procedure were perfect, it would still return the higher-scoring candidate, which in this case is wrong. So the fix should target the scoring function, such as improving the model or training data, rather than the search step.

Key points:
- Compare the score of the correct transcript with the score of the chosen transcript.
- If the correct transcript scores lower, the scoring function is the issue.
- If the correct transcript scores higher but was not chosen, the search procedure is the issue.
- This matches the logic of checking whether optimization or scoring is responsible.

Rubric:
Full credit for identifying the scoring function as the source of the error when the correct transcript receives a lower score, and for explaining that this indicates a ranking problem rather than a search problem.

Use the score comparison to identify the source of the error.

Question:
In one to three sentences, identify and briefly describe the two parts of a common design pattern used to rank candidate outputs.

Sample answer:
The pattern includes an approximate scoring rule that assigns a quality value to each candidate output, and a search or optimization procedure that explores the candidates and returns the one with the best score.

Key points:
- Approximate scoring rule estimates how good each output is
- Search or optimization procedure finds the highest-scoring output

Rubric:
Full credit for naming both the scoring rule and the search/optimization step and briefly explaining each role.

Learn Before

Related