Data Requirements in Model Training Phases
A language model is first trained on a massive dataset containing trillions of words from the public internet. Subsequently, it is adapted to follow user commands and answer questions helpfully using a much smaller, carefully curated dataset of only a few thousand examples. Analyze why the second training phase can be successful with a dramatically smaller dataset compared to the first.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A research lab has access to a large, general-purpose language model that was developed by training it on a vast and diverse collection of text from the internet. The lab's goal is to make this model specialized for generating safe and helpful medical advice. They have a limited budget, allowing them to create only a few thousand high-quality examples of medical questions paired with ideal, safe answers. Which of the following is the most effective and resource-conscious strategy for the lab to pursue?
AI Startup Strategy for Legal Document Summarization
Data Requirements in Model Training Phases