Learn Before
Justifying Scaling Decisions in Multilingual Model Development
A machine learning team is expanding their multilingual language model from supporting 20 languages to over 100. A junior engineer suggests that to save resources, they should keep the model's parameter count and shared vocabulary size the same as the original model. Analyze the potential negative consequences of this approach and explain the underlying reasons why both model size and vocabulary size generally need to increase with the number of supported languages.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A research lab has a successful multilingual model that performs well on 10 distinct languages. The team is now tasked with building a new version to support 100 languages. To manage computational costs, they propose keeping the new model's parameter count (size) and shared vocabulary size identical to the original 10-language model. Based on established scaling principles for such models, what is the most likely outcome of this strategy?
Multilingual Model Development Strategy
Justifying Scaling Decisions in Multilingual Model Development