Construction of the Optimal Sequence in Greedy Search

When generating text one token at a time, a greedy algorithm aims to select the token y_i at step i that maximizes the log-probability of the entire sequence up to that point, log Pr(y_1...y_i | x). This optimization problem can be simplified to choosing the token that maximizes only the conditional log-probability of the current token, log Pr(y_i | x, y_{<i}). Why is this simplification mathematically valid for finding the best current token y_i?

The simplification of the greedy search objective relies on a specific mathematical derivation. Arrange the following steps to correctly represent the logical flow of this derivation, which shows how maximizing the log-probability of the entire sequence up to the current step (log Pr(y_1...y_i | x)) is equivalent to maximizing the log-probability of just the current token (log Pr(y_i | x, y_{<i})).

Explaining Greedy Search Optimization

Construction of the Optimal Sequence in Greedy Search

An autoregressive model generates the token sequence $y = (\text{'The'}, \text{'quick'}, \text{'brown'}, \text{'fox'})$, where $y_1 = \text{'The'}$, $y_2 = \text{'quick'}$, and so on. What does the notation $y_{<3}$ represent in this specific sequence?

True or False: For an autoregressive model generating the output sequence $y = (y_1, y_2, y_3, y_4)$, the notation $y_{<4}$ represents the complete subsequence $(y_1, y_2, y_3, y_4)$.

Formula for Constructing Top-K Candidate Sequences

An autoregressive language model is generating a sequence of tokens, one at a time. To predict the fifth token in the sequence, denoted as $y_5$, the model uses all the previously generated tokens as context. The standard notation for this preceding subsequence of tokens is ____.