Analyzing a Model Development Lifecycle
Analyze the two distinct phases of the model development process described in the case study. For each phase, identify its primary objective and explain how that objective addresses one of the two fundamental challenges of building and using large-scale pre-trained models.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Analyzing a Model Development Lifecycle
A research lab is developing a new foundation model with a limited computational budget. They are considering two primary approaches for the initial training phase:
- Approach 1: Train the model on an extremely large and diverse dataset, incorporating text from the web, academic articles, books, and code, using a general-purpose learning objective.
- Approach 2: Train the model on a smaller, but very high-quality, curated dataset focused on a few key domains (e.g., customer service
Balancing Generalization and Specialization