Search-Procedure Error in Inference
A search-procedure error happens when the scoring model gives the best output a higher score, but the decoding method still returns a different candidate because it does not explore the candidate space well enough. In that situation, the ranking model is not the main problem; the search method is too limited, so the remedy is to improve the search process, such as by widening beam search or using a stronger decoder.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Search-Procedure Error in Inference
Detecting a Problem in the Scoring Step
Score Comparison Diagnostic for Search Errors
When a route-planning system produces a poor route, which two possible failure sources should you separate during debugging?
Diagnosing whether the candidate generator or the reranker caused a missed result helps you decide where to improve a two-stage retrieval system first.
If the approximate search procedure cannot find the value of S that gives the best Score_A(S), the right fix is to improve the _____ procedure.
Match each inference failure type to its correct description in a two-stage prediction system.
Put the debugging steps for an incorrect inference result in the right order.
What is a scoring function problem when diagnosing inference behavior in a machine learning system?
When an inference system gives poor results, randomly choosing between improving retrieval and improving the scoring model is a good debugging strategy.
Choose the missing word in the guidance about fixing a scoring-function problem.
Match each failure cause to the action that best addresses it.
Order the checks for deciding whether a bad prediction comes from the scorer rather than the search process.
How should you respond to search errors versus score-estimation errors in an inference system?
Diagnosing an Incorrect Text Completion from Search Scoring
Why a coin flip is a bad debugging strategy for inference failures
Learn After
A model assigns a higher score to output y_best than to y_found, but the system still returns y_found. What is the most likely issue?
True or False: If a beam search may miss the best candidate, increasing the beam width is a sensible way to reduce that risk.
When a scoring model prefers the correct answer, a _____ can still return a different one if it searches only part of the candidate space.
Match each notation to its meaning when diagnosing a search problem in inference.
Order the steps for telling apart a search problem from a scoring problem.
Why a Better Score Does Not Always Produce the Best Output
Decoding is the likely source of the translation errors.
How can a search method return the wrong result even when its score order is correct?
If a decoder chooses the wrong output even though the score model prefers the right one, what should be improved?
True or False: A search algorithm issue occurs when the search procedure returns the highest-scoring candidate even though a lower-scoring option would be better.