Trade-offs in k-NN Datastore Population
A language model can use an external memory datastore to improve its predictions. One strategy is to populate this datastore with context-to-next-word examples drawn from a massive, general-purpose training corpus. Analyze the primary benefit of this approach compared to using only examples from the current text being generated. Then, identify and explain a significant potential drawback that a system designer would need to consider when implementing this strategy.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Computational Challenge of Large-Scale k-NN Datastores
Datastore Composition in k-NN Language Models
Consider two language models that use an external datastore of (context -> next word) examples to help generate text.
- Model X populates its datastore only with examples from the specific document it is currently generating.
- Model Y's datastore is pre-filled with millions of examples from a vast and diverse library of texts before it begins generating any new document.
When asked to complete a sentence about a niche historical fact not mentioned earlier in the current document, which model is more likely to perform better and why?
Designing a Memory-Augmented Legal AI
Trade-offs in k-NN Datastore Population