Learn Before
Diagram of an Evaluation Model Assessing a Predicted Sequence
The process of assessing a sequence of predicted future values can be represented visually. For example, a sequence of five predicted tokens, , can be shown as input to an evaluation model. This model then utilizes a probability function, denoted as or , to compute a score that reflects the quality or likelihood of the entire predicted sequence.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Evaluating Predicted Sequences
Assessing Predictive Text Systems
A researcher is developing a system that predicts the next five words in a sentence. After the system generates a sequence of five words, denoted as , the researcher wants to use a probabilistic evaluation model, , to assess the quality of the entire generated sequence as a whole. Which of the following expressions correctly represents this evaluation?
When using a probabilistic framework to assess the quality of a predicted sequence of five future values, , the most comprehensive approach is to calculate the probability of each value separately and then average the results.
Diagram of an Evaluation Model Assessing a Predicted Sequence
Learn After
A predictive model generates a sequence of three potential future values: [A, B, C]. Which of the following descriptions best represents a diagram of an evaluation model assessing the quality of this entire predicted sequence?
An engineer is designing a visual aid to explain how an evaluation model assesses a predicted sequence of three tokens: []. The diagram shows the three tokens entering an 'Evaluation Model' box. From this box, three separate arrows point outwards, each leading to a distinct probability score: , , and . What is the primary conceptual error in this diagram?
Diagramming a Sequence Evaluation