Previously attributed to insufficiently flexible function approximation and noise. However, overestimations can occur regardless of the source of approximation error. Since imprecise value estimates are often used for learning, overestimations may be much more common.

University of Michigan - Ann Arbor

University of Toledo

One of the most popular reinforcement learning algorithms, known to sometimes learn unrealistically high action values since it includes a maximization step that tends to prefer overestimated values.

Q-learning

van Hasselt, H., Guez, A., & Silver, D. (2016). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). Retrieved from https://paperswithcode.com/paper/deep-reinforcement-learning-with-double-q

Deep Reinforcement Learning with Double Q-learning

The Problem With Q-learning

DQN is a method using neural network to generate Q-value and select actions instead of forms in original Q-learning method.

Deep Q Networks (DQN)

Double Q learning introduces double estimator to deal with the over-estimation problem. For update of Q values, both estimators use the value of the other estimator. For selecting options, it's still a greedy algorithm based on the average value of two estimators.

Learn Before

Related