Interpreting the SFT Objective Function
Based on the standard formulation of the supervised fine-tuning objective, is the scenario described in the case study plausible? Justify your answer by explaining how the components of the objective function relate to the overall training goal.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is being fine-tuned on a dataset
Dcontaining two input-output pairs:(x1, y1)and(x2, y2). The training objective is to find the model parameters that maximize the sum of the log-probabilities of the concatenated input-output sequences across the entire dataset.Two candidate models, Model A and Model B, produce the following log-probabilities for the concatenated sequences:
- Model A:
log Pr(seq_x1,y1) = -1.2log Pr(seq_x2,y2) = -0.8
- Model B:
log Pr(seq_x1,y1) = -0.9log Pr(seq_x2,y2) = -1.3
Based on the stated training objective, which model is preferred and why?
- Model A:
Interpreting the SFT Objective Function
When fine-tuning a language model with the objective to maximize the sum of log-probabilities across all concatenated input-output sequences in a dataset, which of the following statements accurately describes the training dynamics?