Learn Before
Configuring Memory for Narrative Coherence
Based on the scenario provided, how should the weights (coefficients) in the moving average be configured to best meet the requirement for long-term narrative coherence? Explain your reasoning.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for Memory as a Weighted Moving Average of Keys and Values
Increasing Coefficients as a Heuristic for Weighted Moving Average
A language model's memory component creates a summary vector of past information using a weighted moving average. The weights are determined by a heuristic that assigns significantly higher importance to more recent information. For a task like summarizing a long, complex article, what is the most probable impact of this specific weighting scheme on the model's output?
Learned vs. Heuristic Weights for Memory Summarization
Configuring Memory for Narrative Coherence