A language model is being fine-tuned on a dataset D containing two input-output pairs: (x1, y1) and (x2, y2). The training objective is to find the model parameters that maximize the sum of the log-probabilities of the concatenated input-output sequences across the entire dataset.
Two candidate models, Model A and Model B, produce the following log-probabilities for the concatenated sequences:
- Model A:
log Pr(seq_x1,y1) = -1.2log Pr(seq_x2,y2) = -0.8
- Model B:
log Pr(seq_x1,y1) = -0.9log Pr(seq_x2,y2) = -1.3
Based on the stated training objective, which model is preferred and why?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is being fine-tuned on a dataset
Dcontaining two input-output pairs:(x1, y1)and(x2, y2). The training objective is to find the model parameters that maximize the sum of the log-probabilities of the concatenated input-output sequences across the entire dataset.Two candidate models, Model A and Model B, produce the following log-probabilities for the concatenated sequences:
- Model A:
log Pr(seq_x1,y1) = -1.2log Pr(seq_x2,y2) = -0.8
- Model B:
log Pr(seq_x1,y1) = -0.9log Pr(seq_x2,y2) = -1.3
Based on the stated training objective, which model is preferred and why?
- Model A:
Interpreting the SFT Objective Function
When fine-tuning a language model with the objective to maximize the sum of log-probabilities across all concatenated input-output sequences in a dataset, which of the following statements accurately describes the training dynamics?