• WMT — The majority of language pairs from WMT go through English and the data is from the news domain. We consider data for 13 languages (Ondrej et al., 2017; Bojar et al., 2018; Barrault et al., 2019).

• WAT — The WAT competition covers Asian languages paired with English. We consider data for Burmese-English (Riza et al., 2016), which contains news articles.

WAT contains many other evaluation directions, but many of those are covered by WMT or in a specific domain, so we focus on Burmese-English for WAT only.
• IWSLT — The IWSLT translation competition contains data from TED talks paired with English translations. We use data for 4 languages (Cettolo et al., 2017).

• FLORES — FLORES2(Guzmán et al., 2019) pairs two low resource languages, Sinhala and Nepali, with English in the Wikipedia domain.

• TED — The TED Talks dataset3(Ye et al., 2018) contains translations between more than 50 languages; most of the pairs do not include English. The evaluation data is n-way parallel and contains thousands of directions.

• Autshumato — Autshumato4is an 11-way parallel dataset comprising 10 African languages and English from the government domain. There is no standard valid/test split, so we use the first half of the dataset for valid and second half for test.

• Tatoeba — Tatoeba Challenge5covers 692 test pairs from mixed domains where sentences are contributed and translated by volunteers online. The evaluation pairs we use from Tatoeba cover 85 different languages

list of models

We use publicly available evaluation datasets to evaluate the performance of all of our models. 

To cover our set of 100 languages and 2200 direction

University of Michigan - Ann Arbor

1) Language selection \n 
2) Evaluation
3) Covering the Language Matrix by Mining

How to Build a Many-to-Many Parallel Dataset for 100 Languages

The first step of establishing a Many-to-Many dataset is to select 100 languages for which there already exist high-quality, annotated datasets that can be used for model evaluation.

                 
       Therefore, We cover a diversity of scripts and resource levels (as shown in Table 1) to have high coverage of languages worldwide. (widely spoken language)

Language selection

Supervised translation systems rely on large quantities of parallel sentences

Hence, we use mining technique

Covering the Language Matrix

Goal: 

is to build a single model capable of translating 9, 900 language directions covering 100 languages. 

Challenge: 

This creates several challenges for models with insufficient capacity to capture that many languages and scripts adequatel

Components for Scaling Multilingual Translation Models

It ends with a human evaluation study to understand the quality of our model translations.

We focus on 20 different directions, none of them involving English. We include languages commonly spoken in the same region, such as Japanese-Chinese, Hindi-Tamil, and RussianUkrainian

Learn Before

Related

Learn After