A k-NN Language Model retrieves the 4 nearest neighbors from its datastore for a given query hidden state. The retrieved neighbors, their corresponding token values, and their distances to the query are listed below:
- Neighbor 1: Value = 'cat', Distance = 0.2
- Neighbor 2: Value = 'dog', Distance = 0.3
- Neighbor 3: Value = 'cat', Distance = 0.5
- Neighbor 4: Value = 'fish', Distance = 0.6
Based on this information, what is the aggregated distance, , for the vocabulary token 'cat'?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Softmax-based k-NN Probability Distribution
Calculating Aggregated Distances from Nearest Neighbors
A k-NN Language Model retrieves the 4 nearest neighbors from its datastore for a given query hidden state. The retrieved neighbors, their corresponding token values, and their distances to the query are listed below:
- Neighbor 1: Value = 'cat', Distance = 0.2
- Neighbor 2: Value = 'dog', Distance = 0.3
- Neighbor 3: Value = 'cat', Distance = 0.5
- Neighbor 4: Value = 'fish', Distance = 0.6
Based on this information, what is the aggregated distance, , for the vocabulary token 'cat'?
Analyzing Prediction Outcomes via Neighbor Distances