With the implementation of Double DQN, experimentation with numerous games led to show that the algorithm reduces the initial problem of overestimation. It also shows that the Double DQN algorithm is more accurate with its value estimates and that it has a higher policy quality that is more optimal.

University of Virginia, Charlottesville

University of Toledo

A "Double DQN" system is an implementation of both the Double Q Learning Algorithm and a DQN.

In order to fully implement a Double DQN, two DQN networks would be trained in parallel, similarly to how two tabular datasets are trained in traditional Double Q Learning.

The paper proposing the idea of a Double DQN used stored experience replay values instead to simulate the existence of a secondary network.

As these values were already being used to train the traditional DQN network, the performance benefits are of zero cost in comparison.

Double DQN

van Hasselt, H., Guez, A., & Silver, D. (2016). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). Retrieved from https://paperswithcode.com/paper/deep-reinforcement-learning-with-double-q

Learn Before

Related