Learn Before
A research lab has access to a vast corpus of unlabeled text from the internet but has a very limited budget for creating task-specific labeled datasets. Their goal is to develop a foundational model that can be flexibly adapted to a wide variety of future tasks, often with only a few examples for each new task. Which pre-training paradigm would be the most strategic choice for this lab?
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Selecting a Model Training Strategy
Match each pre-training paradigm with the description that best characterizes its data requirements and common adaptation methods.
A research lab has access to a vast corpus of unlabeled text from the internet but has a very limited budget for creating task-specific labeled datasets. Their goal is to develop a foundational model that can be flexibly adapted to a wide variety of future tasks, often with only a few examples for each new task. Which pre-training paradigm would be the most strategic choice for this lab?