Bilingual alignment addresses the open problem of how to initially align between two languages. Bilingual word embedding allows the NMT system to either start from a word-by-word translation derived from a bilingual word embedding, or initialize the embedding parameters according to bilingual word embedding. Denoising auto-encoder (DAE) can build a shared latent space of two languages by learning to reconstruct sentences in the two languages from a noised version. Unsupervised statistical machine translation has an initial alignment that can be obtained through the back-translated corpora generated by an unsupervised NMT system. Language model pre-training can also be used. 

Bilingual alignment as an unsupervised NMT component to exploit monolingual data 

Translation quality needs to be improved based on the initial alignment, where iterative back translation is commonly used. Filtering out low-quality pseudo parallel sentence pairs is a good method. Another method is to add a term in training objectives to avoid forgetting the alignment from bilingual word embeddings during the training. Furthermore, unsupervised SMT can also be used to improve the iterative back-translation. This can be done by leveraging both unsupervised SMT and SMT for back translation and then train the NMT models with the pseudo parallel data from this approach. SMT can also act as a posterior regularization to denoise the parallel data generated by NMT systems.

Translation improvement as an unsupervised NMT component to exploit monolingual data

Unsupervised NMT Components
Unsupervised NMT relies on two components to ensure learning efficiency and quality:
- Bilingual alignment
- Translation improvement


Columbia University

Unsupervised NMT deals with the zero-resource translation scenario where there are no parallel sentences whatsoever. It uses only monolingual data to get an initial alignment and improve the translation via iterative back translation.

Unsupervised NMT for leveraging monolingual data for Low-Resource NMT

Wang, R., Tan, X., Luo, R., Qin, T., & Liu, T.-Y. (8 2021). A Survey on Low-Resource Neural Machine Translation. Στο Z.-H. Zhou (Επιμ.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21 (σσ. 4636–4643). doi:10.24963/ijcai.2021/629


Learn Before

Related

Learn After