Multilingual language models (LMs) are typically trained in two phases: a “pre-training” phase, in which unlabeled data from one or multiple languages may be used, and a “fine-tuning” phase, in which task-specific labeled data is used to build a system capable of solving a target task.