1. Allows updates for a wide range of the state-action space.
2. Less variance
3. Reduces risk of instabilities

San Diego State University

One value function we frequently used in value-based methods is the action-value function Q(s,a), which represents the total value of taking action a in state s. It is the sum of the future rewards r, adjusted by a discount factor gamma.
$$Q^{*}(s, a)=\max _{\pi} \mathbb{E}\left[r_{t}+\gamma r_{t+1}+\gamma^{2} r_{t+2}+\ldots \mid s_{t}=s, a_{t}=a, \pi\right]$$

The basic steps of deep Q-learning algorithms:
1. Train convolutional neural network to extract the essential features that can help the agent make the decision.
2. Calculate the Q-Value of each possible action.
3. Perform back-propagation to find the most accurate Q-Values.

Learn Before

Related