Learn Before
Selecting a Model Training Strategy
A research team aims to develop a model that can answer questions about a highly specialized and new scientific domain. They have collected a massive corpus of research papers from this domain, but none of it is in a question-and-answer format. The team has the resources to manually create a small, high-quality dataset of 1,000 question-answer pairs. Given the available data and the team's goal, which combination of initial training and subsequent adaptation methods would be most effective and resource-efficient? Justify your choice.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Selecting a Model Training Strategy
Match each pre-training paradigm with the description that best characterizes its data requirements and common adaptation methods.
A research lab has access to a vast corpus of unlabeled text from the internet but has a very limited budget for creating task-specific labeled datasets. Their goal is to develop a foundational model that can be flexibly adapted to a wide variety of future tasks, often with only a few examples for each new task. Which pre-training paradigm would be the most strategic choice for this lab?