Learn Before
Evaluating a Distributed Training Strategy
Critically evaluate the team's decision to use only data parallelism. Will this strategy succeed in training the model? Justify your answer by explaining the fundamental limitation of their chosen approach in this specific context, and propose a more suitable strategy or combination of strategies.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating a Distributed Training Strategy
A machine learning team is training a language model whose parameters are too large to fit into the memory of a single accelerator. Which of the following statements most accurately analyzes the trade-offs between the primary strategies they could employ to address this specific problem?
Match each distributed training strategy with the primary computational challenge it is designed to address when training very large models.