Learn Before
Dense vs. Sparse Rewards
Reinforcement learning feedback can be categorized based on its frequency. Dense rewards are provided immediately and frequently, which generally makes policy training easier and more efficient. In contrast, sparse rewards are given only upon task completion. While dense feedback is often preferred, many scenarios, particularly in NLP, are inherently structured with sparse rewards.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Dense vs. Sparse Rewards
Reward Shaping as a Solution for Sparse Rewards
Transforming Sparse Rewards into Dense Supervision Signals
An AI is being trained to generate a multi-paragraph summary of a long document. The AI writes the summary one sentence at a time. A quality score is given only after the entire summary is complete. For each individual sentence generated before the final one, the score is zero. What is the most significant learning difficulty the AI will face due to this scoring method?
Training an Agent for a Text-Based Game
Credit Assignment in AI Poetry Generation
Methods for Mitigating Sparse Rewards
Learn After
An AI agent is being trained to play a complex board game. The agent only receives a reward signal at the very end of the game: +1 for a win, -1 for a loss, and 0 for a draw. No feedback is given for any of the individual moves made during the game. Which of the following best describes this reward structure and its primary challenge for training the agent?
Reward System Design for a Summarization Agent
Comparing Reward Structures in AI Training