Selecting a Pre-trained Model for a Low-Resource Language
A development team is building a named-entity recognition tool for Welsh, a language with limited available training data. They have two pre-trained multilingual models to choose from for fine-tuning. Analyze the two options below and determine which model is the better choice, justifying your answer based on principles of knowledge transfer.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A machine learning team is tasked with creating a text classification model for the Malagasy language, which has a very small amount of available training data. The team decides to leverage a large, pre-trained multilingual model and then fine-tune it on their limited Malagasy dataset. To maximize the effectiveness of this approach, which pre-training strategy for the multilingual model should they prioritize?
Selecting a Pre-trained Model for a Low-Resource Language
Critique of a Model Training Strategy