Learn Before
An auto-regressive model is processing the input sequence of tokens: ['The', 'cat', 'sat']. When the model uses the prefix ['The', 'cat'] to generate the next token, 'sat', what is the content of the corresponding Key-Value (KV) cache state that is created at this step?
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An auto-regressive model is processing the input sequence of tokens:
['The', 'cat', 'sat']. When the model uses the prefix['The', 'cat']to generate the next token,'sat', what is the content of the corresponding Key-Value (KV) cache state that is created at this step?An auto-regressive model is generating a series of Key-Value (KV) cache states for the input sequence of tokens:
['The', 'quick', 'brown']. Arrange the following events in the correct chronological order in which they occur during this process.Prefix Cache Reuse Scenario