Search Errors in LLMs
A search error occurs when a language model's search or decoding process fails to generate any high-quality candidates in its output list. This issue is indicated when an oracle model re-ranks the N-best list but finds only a small performance gain, suggesting that no superior options were available to be selected.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Related
Language Model Performance Diagnosis
A development team is analyzing the performance of their language model. For a set of test prompts, they take the top 5 responses generated by their model and have a significantly more powerful, 'oracle' model select the best response from that list. They find that the average quality score of their model's original top-ranked response is 70%, while the average quality score of the response selected by the oracle model is 95%. What does this large performance gap most strongly suggest about the primary limitation of the team's model?
Interpreting Model Diagnostic Results
Search Errors in LLMs