Notational Simplification in Fine-Tuning Formulas
For the sake of clarity and to keep mathematical notation uncluttered, the parameters being adjusted during fine-tuning, which are initialized from a pre-trained state , are often simplified in formulas to just . Although the superscript '+' is omitted in this convention, it is crucial to remember that these parameters originate from the pre-trained model and are not randomly initialized.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Related
Notational Simplification in Fine-Tuning Formulas
A language model is being fine-tuned on a dataset of customer support chat logs to improve its ability to generate helpful responses. The training process is guided by the objective function: During one step of this process, the model processes a single
(query, response)pair from the dataset. What is the role of the specific componentlog Pr(response|query)for this single pair?The following equation represents the primary goal of a common model training process. Match each mathematical symbol from the equation to its correct description.
Evaluating an Alternative Fine-Tuning Objective
Token-Level Conditional Log-Probability in Supervised Fine-Tuning
Learn After
A researcher is fine-tuning a pre-trained language model on a new dataset. They represent the optimization objective using the following simplified notation:
Based on standard conventions in this field, what is the most accurate interpretation of the parameters
θbeing optimized in this formula?When the supervised fine-tuning objective is written as , the parameters denoted by are typically initialized from a random distribution before the optimization process begins.
Potential Misinterpretation of Fine-Tuning Notation