Learn Before
A masked language model is given the input sequence: 'The quick brown [MASK] jumps over the lazy dog.' The original, unmasked token at the [MASK] position was 'fox'. Two different versions of the model, Model A and Model B, are used to predict the masked token.
- Model A assigns a probability of 0.85 to the token 'fox'.
- Model B assigns a probability of 0.15 to the token 'fox', and its highest predicted probability is 0.40 for the token 'cat'.
Based on the probability assigned to the *corre
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A masked language model is given the input sequence: 'The quick brown [MASK] jumps over the lazy dog.' The original, unmasked token at the
[MASK]position was 'fox'. Two different versions of the model, Model A and Model B, are used to predict the masked token.- Model A assigns a probability of 0.85 to the token 'fox'.
- Model B assigns a probability of 0.15 to the token 'fox', and its highest predicted probability is 0.40 for the token 'cat'.
Based on the probability assigned to the *corre
Analyzing Model Learning via Token Probability
A language model is being trained on the task of filling in masked words. At an early stage of training, for the sentence 'The sun rises in the [MASK]', the model assigns a probability of 0.05 to the correct word 'east'. After many more rounds of successful training on a large dataset, the model is presented with the same masked sentence. Which of the following outcomes is the most plausible and directly reflects the objective of this training process?