Learn Before
Structuring a Sample from Input and Output Segments
During fine-tuning, each data sample from the tuning dataset (denoted as ) is conceptually divided into two distinct components: an input segment, represented as , and an output segment, represented as . These segments are then structured as a single sequence for model processing, typically by concatenating them, which is formally expressed as: .
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Dataset Composition for RL Fine-Tuning in RLHF
A machine learning engineer is creating a dataset to fine-tune a language model to act as a helpful assistant. The goal is to teach the model to follow instructions and provide complete, high-quality answers. Which of the following examples represents the most effective input-output pair for this supervised fine-tuning task?
Structuring a Sample from Input and Output Segments
Deconstructing an SFT Training Sample
Constructing an SFT Training Pair for Text Summarization
Learn After
Selective Gradient Propagation for Sub-sequence Loss
Sample-wise Negative Log-Likelihood Loss for a Sub-sequence
For a supervised fine-tuning task, a single training instance consists of an input segment (
xsample) and a corresponding output segment (ysample). Ifxsampleis 'Instruction: Translate to Spanish. Input: Hello.' andysampleis 'Response: Hola.', which of the following represents the correct structure for the final combined sample that the model will process?Deconstructing a Fine-Tuning Sample
In preparing a data sample for supervised fine-tuning, a common practice is to structure the sample by concatenating the output segment (
ysample) and the input segment (xsample) into a single sequence:sample = [ysample, xsample]. What is the primary reason for placing the output segment before the input segment in this structure?