A research team is planning to train a new language model. Their previous model had 1 billion parameters and was trained on 100 billion tokens of text. For their new project, they plan to increase the model size to 10 billion parameters and the training dataset to 1 trillion tokens. Which statement best analyzes the expected change in computational resource requirements for this new project?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Distributed Systems for LLM Training Efficiency
Feasibility Analysis of a Model Training Plan
A research team is planning to train a new language model. Their previous model had 1 billion parameters and was trained on 100 billion tokens of text. For their new project, they plan to increase the model size to 10 billion parameters and the training dataset to 1 trillion tokens. Which statement best analyzes the expected change in computational resource requirements for this new project?
Analyzing the Drivers of Computational Cost in Model Training