The respective strengths of the model-free versus model-based approaches depend on different factors.

• First, the best suited approach depends on whether the agent has access to a model of the environment. If that’s not the case, the learned model usually has some inaccuracies that should be taken into account.
• Second, a model-based approach requires working in conjunction with a planning algorithm (or controller), which is often computationally demanding. The time constraints for computing the policy π(s) need to be taken into account.
• Third, for some tasks, the structure of the policy (or value function) is the easiest one to learn, but for other tasks, the model of the environment may be learned more efficiently due to the particular structure of the task. Thus, which one performs better depends on the structure of the model, policy, and value function