Bi-directional NMT trains both translation directions in the same model by using a tag that indicates the direction at the beginning of source sentences. Afterward, it leverages both source-side and target-side monolingual data through back and forward translation.

Google

Considering that both the source and target sides may have monolingual data that has valuable information, one can leverage both at once via joint training on the two translation directions. This type of dual learning simultaneously improves the two models on both translation directions by aligning the original monolingual sentences $$x$$ and the sentences $$x'$$ translated forward and then backward ($$x \to y' \to x'$$) by the two models. Dual learning can be further improved by introducing multi-agent frameworks for both translation directions. Specific approaches for joint training include iterative back translation, bi-directional NMT, and mirror-generative NMT.

Joint Training on Both Translation Directions for Leveraging Monolingual Data for Low-Resource NMT

Wang, R., Tan, X., Luo, R., Qin, T., & Liu, T.-Y. (8 2021). A Survey on Low-Resource Neural Machine Translation. Στο Z.-H. Zhou (Επιμ.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21 (σσ. 4636–4643). doi:10.24963/ijcai.2021/629


A Survey on Low-Resource Neural Machine Translation

Iterative back translation is a method of joint training that simultaneously trains neural machine translation (NMT) models on both translation directions. It iteratively updates the back-translated corpus using the increasingly improved NMT models.

Iterative Back Translation in Joint Training for Low-Resource NMT

Bi-directional NMT for Joint Training in Low-Resource NMT

Mirror-generative NMT jointly trains translation models on both directions, alongside the language models for both the source and target languages, by utilizing a shared latent variable.

Learn Before

Related