Learn Before
Calculating Risk for a Candidate Sentence
A language model is tasked with generating a one-sentence summary. To select the best option from several candidates, it uses a framework where a risk function, , quantifies the penalty for choosing a candidate summary, , when a reference summary, , is considered the ground truth. The specific risk function is defined as: , where is a similarity score calculated as: (Number of unique words common to both summaries) / (Total number of unique words in the reference summary, ). Based on the information provided below, calculate the risk value of choosing the candidate summary.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Calculating Risk for a Candidate Sentence
A team is developing a text summarization system. The primary goal is to generate summaries that are factually consistent with the source document, even if it means the summary is less fluent or grammatically perfect. The system generates multiple candidate summaries and must select the best one by minimizing the expected penalty against all other candidates. Which of the following penalty functions, R(y, y_r), which measures the cost of choosing candidate summary 'y' over reference summary 'y_r', would be most appropriate for this specific goal?
A crucial aspect of selecting the best output from a set of candidates is defining a function, R(y, y_r), that measures the penalty of selecting a candidate output 'y' over another potential output 'y_r'. Match each description of a penalty function to the primary type of error it is designed to penalize.