Learn Before
Training Objective as Joint Log-Likelihood Maximization of Concatenated Sequences
A training objective for a model can be formulated as maximizing the joint log-likelihood of concatenated input-output sequences. For a dataset D of input-output pairs (x, y), the optimal parameters ˜θ are found by maximizing the sum of the log-probabilities of the combined sequence seq_{x,y}. The formula is: This approach is equivalent to maximizing the conditional log-likelihood log Pr(y|x) when the input distribution Pr(x) is not dependent on the model parameters θ.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Fine-Tuning Objective as Log-Likelihood Maximization
Training Objective as Joint Log-Likelihood Maximization of Concatenated Sequences
A machine learning engineer is fine-tuning a pre-trained language model on a specialized dataset of question-answer pairs. The chosen training objective is to adjust the model's parameters to maximize the sum of the log-probabilities of the ground-truth answers, conditioned on their corresponding questions. Which statement best analyzes the direct effect of this training objective on the model's behavior?
Interpreting Fine-Tuning Loss
Analyzing Fine-Tuning Behavior
When fine-tuning a language model, the objective of maximizing the sum of the log-likelihoods of the true responses given the prompts is mathematically equivalent to minimizing the mean squared error loss over the dataset.
Learn After
A language model is being trained on a dataset of input-output pairs
(x, y). Two different training objectives are proposed:- Objective A: Maximize the sum of
log Pr(y|x)over all pairs in the dataset. - Objective B: Maximize the sum of
log Pr(sequence)over all pairs, wheresequenceis the concatenation ofxandy.
Under which of the following conditions will optimizing for Objective B be mathematically equivalent to optimizing for Objective A?
- Objective A: Maximize the sum of
Equivalence of Training Objectives
Evaluating a Language Model's Training Objective