Calculating Output Risk in a Generation Task
A language model is given an input sentence and generates a set of three possible output sentences. Your task is to calculate the total risk for the candidate output 'The cat sat on the mat' using the provided data and the general formula for risk. The formula is:
Risk(y) = Σ [R(y, yr) * Pr(yr|x)] over all yr in the set of possibilities.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Calculating Output Risk in a Generation Task
A system calculates the total risk for a candidate output 'y' against a set of reference outputs 'yr' using the formula: Risk(y) = Σ [R(y, yr) * Pr(yr|x)], where R(y, yr) is a risk function and Pr(yr|x) is the probability of a reference output. Assume the risk function R(y, yr) returns a value of 1 if y is different from yr, and 0 otherwise. If the probability, Pr(yr|x), of a specific reference output 'yr' that is different from the candidate 'y' increases, what is the direct impact on the calculated Risk(y)?
Role of Probability Weighting in Risk Calculation