Learn Before
A language model enhanced with a nearest-neighbor mechanism needs to find relevant information from its external datastore to help predict the next word. Arrange the following steps in the correct chronological order to describe how the model retrieves this information.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Comprehension in Revised Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Using Reference Tokens to Define a Vocabulary Distribution in k-NN LM
Role of Internal State in Datastore Search
A language model enhanced with a nearest-neighbor mechanism needs to find relevant information from its external datastore to help predict the next word. Arrange the following steps in the correct chronological order to describe how the model retrieves this information.
A language model enhanced with a nearest-neighbor search mechanism is generating text. The model's current internal state, representing the prefix 'The scientist made a groundbreaking...', is used as a query to search an external datastore. The datastore contains pairs of (context representation, associated word). If the search retrieves the three words 'discovery', 'advance', and 'finding' as reference tokens, which statement most accurately describes how these specific words were selected?