Learn Before
Correcting Datastore Construction in a k-NN Language Model
A developer is creating a datastore for a k-nearest-neighbor language model by processing the sentence 'The cat sat on the mat'. For the token 'sat', they create a key-value pair where the key is the model's final hidden state vector at the position of 'sat', and the value is the token 'sat' itself. Analyze why this construction is incorrect and describe the correct value that should be paired with this key.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is processing the text 'The quick brown fox jumps'. To build a datastore for a nearest-neighbor component, it must create a key-value pair
(z, w)for the context at each token. For the token 'brown', what would constitute the correct key-value pair?Correcting Datastore Construction in a k-NN Language Model
Analyzing Datastore Key-Value Pair Creation