Learn Before
Relationship Between Decoding Networks for Inference
In the context of preparing a language model for autoregressive generation, an input sequence x is processed by a function denoted as Dec_kv(x) to populate a cache. This function is architecturally identical to the model's standard decoding network, Dec(x). Given this information, explain the key functional difference between Dec_kv(路) and Dec(路) by describing what each function is configured to output.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Layer-wise Structure of the KV Cache
A large language model processes an input prompt, denoted as
x, using a functionDec_kv(x)as part of its inference process. This function utilizes the model's standard decoding network but is configured for a specific preparatory task. Based on this context, what is the primary output of theDec_kv(x)function?In the context of prefilling a Key-Value cache for an input prompt, the function
Dec_kv(路)represents a neural network with a fundamentally different architecture than the standard decoding network,Dec(路), as it is specialized solely for computing key-value pairs.Relationship Between Decoding Networks for Inference