Formula

Computational Cost of Exhaustive Search in Sequence-to-Sequence Models

While exhaustive search guarantees finding the optimal sequence, it incurs a prohibitive computational cost of O(YT)\mathcal{O}(|\mathcal{Y}|^{T'}), where Y|\mathcal{Y}| is the vocabulary size and TT' is the sequence length. This exponential growth makes it practically infeasible; for instance, evaluating a sequence of length T=10T'=10 with a vocabulary of Y=10000|\mathcal{Y}|=10000 would require assessing 1000010=104010000^{10} = 10^{40} sequences, which is beyond the capabilities of foreseeable computers.

0

1

Updated 2026-05-14

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L