Learn Before
Evaluating a Prediction Strategy
A team is developing a language model for two distinct tasks. For Task A, a medical information bot, the model must provide the most factually accurate and reliable information. For Task B, a creative story generator, the model needs to produce diverse and interesting narratives. The team implements a prediction rule for both tasks where the model always selects the single word with the highest calculated probability to be the next word in a sequence.
Analyze the suitability of this prediction rule for both Task A and Task B. Explain why this rule might be highly effective for one task but lead to undesirable results in the other.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Using Optimized Predictions as Learning Targets
General Formula for Prediction via Maximum Probability
A language model is given the input 'The ocean is...' and calculates the conditional probability for four candidate words to be the next word. Based on the values below, which word would a model that predicts by maximizing probability choose?
- P("deep" | "The ocean is...") = 0.75
- P("cold" | "The ocean is...") = 0.15
- P("running" | "The ocean is...") = 0.09
- P("quiet" | "The ocean is...") = 0.01
Evaluating a Prediction Strategy
Spam Filter Classification