University of Michigan - Ann Arbor

On-policy methods attempt to evaluate or improve the policy that is used to make decisions. In contrast, off-policy methods evaluate or improve a policy different from that used to generate the data. Classicla on-policy methods include Sarsa Algorithm. As a comparison, Q-learning method is a classical off-policy method.

On-policy vs Off-policy

Here is a free pdf version of a book about Reinforcement Learning:
http://incompleteideas.net/book/RLbook2020.pdf

Useful Book for Reinforcement Learning

Sarsa Algorithm

Basic Q-learning Method 

Since MuZero uses a replay buffer, one can consider it off-policy. However, viewed from the perspective of the behavior and training policies, it could also be considered on-policy.

Learn Before

Related