A system for generating creative story endings uses the formula to select the best ending from a set of candidates. The verifier function, , is designed to score endings based solely on their grammatical correctness. The system consistently selects endings that are grammatically perfect but are reported by users as being dull and unoriginal. What is the most likely reason for this suboptimal outcome?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A system generates a set of four candidate responses (
y) to a user's query. A scoring function,V(y), evaluates each response and assigns it a score from 0 to 1, where a higher score indicates a better response.Candidate Response Score V(y)Response A 0.78 Response B 0.91 Response C 0.65 Response D 0.89 Given the selection formula
ŷ = argmax(y ∈ Dc) V(y), whereDcis the set of all candidate responses, which response would be selected as the best candidate,ŷ?A system for generating creative story endings uses the formula to select the best ending from a set of candidates. The verifier function, , is designed to score endings based solely on their grammatical correctness. The system consistently selects endings that are grammatically perfect but are reported by users as being dull and unoriginal. What is the most likely reason for this suboptimal outcome?
Evaluating Verifier Function Design