Learn Before
Auto-regressive Decoding in Machine Translation
In the context of machine translation, the auto-regressive decoding process generates each token of the translated text (target language) by conditioning on both the complete source language sequence and the tokens that have already been generated in the target language.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Beam search
Auto-regressive Decoding in Machine Translation
An autoregressive sequence generation model is tasked with producing an output. At each step, it calculates the probability for every possible next element and selects the single element with the highest probability before moving to the next step. What is the primary limitation of this step-by-step selection strategy?
Decoder Input Analysis
Diagnosing Translation Degradation
Learn After
A sequence-to-sequence model is translating a sentence from one language to another. At the third step of generating the output, it incorrectly predicts a word. Subsequent words generated by the model are also incorrect, leading to a final translation that is grammatically flawed and semantically nonsensical, even though the initial error was minor. Which characteristic of the generation process is the most likely cause of this cascading failure?
Diagnosing Long-Range Dependency Issues
Trade-offs in Translation Generation