Fine-Tuning Pre-trained Models for Downstream Tasks

Instruction Fine-Tuning

Superficial Alignment Hypothesis

Challenge of Opaque Pre-Training Data in Fine-Tuning

A team develops a large language model pre-trained on a massive, diverse corpus of text from the internet. When initially tested on the task of generating concise summaries of legal documents, its performance is poor and unstructured. The team then collects a small, curated dataset of 500 legal documents and their corresponding expert-written summaries. After training the model on this small dataset, its ability to summarize new legal documents improves dramatically. Which statement best analyze

Critiquing a Model Training Hypothesis

Implicit Learning of Instruction-Response Mappings During Pre-training

Explaining the Impact of Targeted Training