We group languages based on two criteria: the amount of training data and their vocabulary. 

The motivation for these criteria is that we can learn a specific layer for a language with enough data, and for the rest, overlapping vocabulary is a good proxy for similar languages.

University of Michigan - Ann Arbor

a layer whose parameters are split by language or language group based on similarity in vocabulary.

Each translation direction only accesses a subset of these parameters, allowing the model capacity to scale without significantly affecting the training and inference time.

Learn Before

Related