The primary purpose of the supervisory phase that follows pre-training is to introduce entirely new capabilities, such as the ability to summarize text, which the model did not acquire in any form during its initial, large-scale training.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A research team develops a large language model by training it on a massive corpus of text from the internet. When they give the model the instruction, 'Translate the following English sentence to French,' the model instead continues the sentence in English with a grammatically correct but irrelevant phrase. However, after a second, much shorter training phase using a small, curated dataset of English-to-French sentence pairs, the model correctly performs the translation task. Which of the follo
Fine-Tuning as a Mechanism for Activating Pre-Trained Knowledge
Evaluating a Researcher's Conclusion on Model Training
The primary purpose of the supervisory phase that follows pre-training is to introduce entirely new capabilities, such as the ability to summarize text, which the model did not acquire in any form during its initial, large-scale training.