Learn Before
Value Loss Calculation for a Single Sequence
An A2C agent is being trained to process a single text document that consists of 250 tokens. To update the value network, the squared temporal difference (TD) error is calculated at each of the 250 steps. The final loss is computed by summing these 250 squared errors and then dividing the sum by a specific number to find the average. What is this number, and what does it represent in the general value loss formula?
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
An engineer is training a reinforcement learning agent to process text. The agent receives a single sequence of 64 tokens as input. To update its value network, the engineer calculates the squared temporal difference error for each of the 64 token-processing steps. The final loss is the average of these 64 squared errors. In the formula for the average, what number should the sum of the squared errors be divided by?
A2C Value Loss for Variable-Length Sequences
Value Loss Calculation for a Single Sequence