Recursive Formula for Cumulative Average
A cumulative average can be calculated using a recursive formula. This means the average at the current step can be computed based on the value of the average from the preceding step and the current data point, which is more efficient than recalculating the sum over the entire history at each step.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Recursive Formula for Cumulative Average
A language model's memory component is designed to create a summary vector at each step by calculating the average of all key-value pairs from the start of the sequence up to that current step. When this model is processing a very long sequence, what is the effect on the summary vector's representation of information from the very beginning of the sequence as the model approaches the end?
Analysis of Memory Summary Techniques
Selecting a Memory Summarization Strategy
Formula for Memory as a Cumulative Average of Keys and Values
Learn After
Recursive Formula for Memory as a Cumulative Average
A real-time analytics system tracks the average score for a popular online game. After 1,000,000 games have been played, the system has calculated a cumulative average score. When the 1,000,001st game score is recorded, which of the following methods for updating the average is the most computationally efficient and mathematically correct?
Real-time Performance Monitoring
Calculating an Updated Cumulative Average