Linear Interpolation of k-NN and LLM Distributions
The final output probability distribution in a -NN language model is formed by combining the retrieval-based distribution from the -nearest neighbors with the standard output distribution of the base large language model (LLM). This combination is performed using a linear interpolation, regulated by a coefficient , which adjusts the relative weight of the two distributions.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Aggregated Distance Calculation for k-NN Vocabulary Distribution
Linear Interpolation of k-NN and LLM Distributions
Characterizing a Retrieval-Based Probability Distribution
A k-Nearest Neighbors Language Model (k-NN LM) is generating text and needs to predict the next token. It queries its datastore and retrieves the 5 nearest reference tokens, along with their corresponding distances: {"river": 0.1}, {"stream": 0.2}, {"river": 0.3}, {"ocean": 0.8}, {"river": 0.9}. How are these retrieved tokens and their distances used to construct a new probability distribution over the model's vocabulary?
Evaluating a k-NN LM's Intermediate Output
Learn After
k-NN LM Interpolation Formula
A developer is working with a language model that generates text by combining its own internal predictions with a probability distribution derived from similar examples in a large external datastore. The developer observes that the model's output is often creative but frequently ignores the specific, factual information present in the retrieved examples. To make the model's output adhere more closely to the facts in the datastore, how should the developer adjust the interpolation coefficient that balances these two distributions?
Analyzing Extreme Interpolation Scenarios
Diagnosing a Retrieval-Augmented Chatbot