Learn Before
Effect of Averaging on Prediction Confidence
Consider a scenario where two different sets of instructions are given to a language model for the same classification task. The first set of instructions results in a high-confidence probability distribution (e.g., {Class A: 0.9, Class B: 0.1}), while the second results in a lower-confidence distribution that still favors the same class (e.g., {Class A: 0.6, Class B: 0.4}). If you combine these outputs by taking a simple average of the probabilities for each class, what is the likely effect on the final prediction's confidence level for Class A compared to the most confident individual output? Explain your reasoning.
0
1
Tags
Data Science
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is used for a sentiment classification task. To improve reliability, two different instructions are given to the model for the same input text, resulting in two sets of output probabilities for the possible classes (Positive, Neutral, Negative).
- Output from Instruction 1: {Positive: 0.7, Neutral: 0.2, Negative: 0.1}
- Output from Instruction 2: {Positive: 0.5, Neutral: 0.4, Negative: 0.1}
If you combine these two outputs by taking the simple average of their probabilities for each class, what is the final combined probability for the 'Positive' class?
A language model is used for a sentiment classification task. To improve reliability, three different instructions are given to the model for the same input text, resulting in three sets of output probabilities for the classes 'Positive', 'Neutral', and 'Negative'. The outputs are as follows:
- Output 1: {Positive: 0.6, Neutral: 0.3, Negative: 0.1}
- Output 2: {Positive: 0.7, Neutral: 0.2, Negative: 0.1}
- Output 3: {Positive: 0.5, Neutral: 0.4, Negative: 0.1}
If the final prediction is determined by taking a simple average of the probabilities for each class across all three outputs, what will be the final predicted class?
Effect of Averaging on Prediction Confidence