Learn Before
Decoder Input Analysis
An autoregressive decoder generates an output sequence one element at a time. At each step, the decision for the next element is typically conditioned on three key inputs: 1) the element generated in the previous step, 2) the decoder's current hidden state, and 3) the context derived from the entire input sequence. Explain the distinct purpose of each of these three inputs in this process.
0
1
Tags
Data Science
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Beam search
Auto-regressive Decoding in Machine Translation
An autoregressive sequence generation model is tasked with producing an output. At each step, it calculates the probability for every possible next element and selects the single element with the highest probability before moving to the next step. What is the primary limitation of this step-by-step selection strategy?
Decoder Input Analysis
Diagnosing Translation Degradation