A memory component in a transformer-based model is designed to create a summary by computing the simple, unweighted average of the last 10 key-value pairs. Which statement accurately describes a fundamental property of this specific summarization method?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Formula for Memory as a Moving Average of Keys and Values
Example of a Moving Average-based Cache
Cumulative Average of Keys and Values for Memory Component
Calculating a Memory Component Summary
When using a moving average of the last
nkey-value pairs to create a single summary vector for a memory component, what is the primary effect of significantly increasing the window sizen?Weighted Moving Average for Memory Component
A memory component in a transformer-based model is designed to create a summary by computing the simple, unweighted average of the last 10 key-value pairs. Which statement accurately describes a fundamental property of this specific summarization method?