Targeted SFT Data Curation for Stylistic Control
A team is fine-tuning a large language model to be a concise and helpful assistant. They notice that while the model's answers are factually correct, they are often overly long and fail to adhere to specific formatting requests (like using bullet points). Describe one advanced data construction technique the team could implement in their next fine-tuning dataset to specifically address these two issues, and explain the mechanism by which this technique would improve the model's performance.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Computing Sciences
Foundations of Large Language Models Course
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Optimizing a Fine-Tuning Data Pipeline
A development team is fine-tuning a language model to act as a programming assistant. Their initial training data consists of thousands of simple instruction-code pairs, such as a prompt asking for a function to add two numbers and the corresponding correct code. After training, they observe that the model performs well on simple, one-step tasks but consistently fails to generate correct code for complex problems that require breaking the problem down into multiple logical steps. Which of the following advanced data construction strategies is most likely to address this specific performance issue?
Targeted SFT Data Curation for Stylistic Control