Learn Before
Evaluating a Simple Risk Function
In a system that generates text, a simple risk function is proposed: R(y, yr) = 0 if the candidate output y is exactly identical to the reference output yr, and R(y, yr) = 1 otherwise. Analyze the primary limitation of using this function for tasks where semantic similarity is more important than exact string matching.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Risk of an Output in Minimum Bayes Risk Decoding
A language model is generating a summary for a document. The ideal, human-written reference summary is
yr: "The study found a significant link between diet and health."The model considers two candidate summaries:
y1: "The study found a significant link between diet and health."y2: "Research showed a connection between food and wellness."
Given the purpose of a risk function,
R(candidate, reference), which of the following statements best describes how it would operate in this scenario?Evaluating a Simple Risk Function
Evaluating a Risk Function for Legal Translation