Google

A Supervised Fine-Tuning (SFT) dataset, commonly denoted by $$S$$, is a curated collection of annotated input-output pairs used to adapt pre-trained language models. In each pair, the input consists of a user instruction, and the output is the corresponding correct or 'gold-standard' response that the model is expected to generate.

Definition of SFT Datasets

A Supervised Fine-Tuning (SFT) dataset is composed of samples, where each sample includes an input `x` and an output `y`. The input `x` typically combines an instruction with user-provided context (e.g., an article to summarize, a report to analyze). The output `y` is the desired response that correctly follows the instruction. For instance, tasks can range from summarization and financial data extraction to email classification and technical troubleshooting.

Example of SFT Dataset Samples

The effectiveness of an SFT dataset is determined by its quantity, quality, and relevance to the LLM's intended tasks. The final performance of the language model is highly dependent on the quality of this data, which underscores the need for its careful development and evaluation.

Key Attributes of Effective SFT Datasets and Their Impact on LLM Performance

In Supervised Fine-Tuning, training data is structured as pairs of sequences. The input sequence, denoted as $x = x_0...x_m$, typically consists of a combination of an instruction and user-provided context. The corresponding output sequence, denoted as $y = y_1...y_n$, represents the desired response that the model is trained to generate.

Input and Output Sequences in SFT

A team is preparing a dataset to fine-tune a pre-trained language model to follow specific instructions. Which of the following data entries best exemplifies the fundamental structure of a single sample in a Supervised Fine-Tuning (SFT) dataset?

Analyze the following scenario and determine if the described dataset is immediately suitable for its intended purpose. Justify your reasoning based on the core principles of constructing such datasets.

Evaluating a Potential Fine-Tuning Dataset

The datasets used for Supervised Fine-tuning (SFT) are distinct from those used in pre-training. While they are typically much smaller in size, they are highly specialized, containing curated pairs of task-specific instructions and their corresponding expected outputs.

Characteristics of SFT Datasets

You are reviewing a dataset intended for training a language model to follow instructions. Below are three data samples. Identify which sample (A, B, or C) is NOT structured as a standard input-output pair for this purpose and briefly explain why it is unsuitable.

**Sample A:**
*   **Input:** "Instruction: Summarize the following text in one sentence. Text: The sun is a star at the center of the Solar System. It is a nearly perfect sphere of hot plasma, with internal convective motion that generates a magnetic field via a dynamo process."
*   **Output:** "The sun is a plasma star at the center of our solar system that generates a magnetic field."

**Sample B:**
*   **Input:** "Instruction: Write a short poem about the ocean."
*   **Output:** A pair of responses is provided for comparison: Response 1 ('The waves crash high, a salty sigh, beneath the endless, azure sky.') is marked as 'better' than Response 2 ('Water is blue and very deep.').

**Sample C:**
*   **Input:** "Instruction: Translate 'hello world' to French."
*   **Output:** "Bonjour le monde"

Learn Before

Related