Neural network needs a lot of data for small steps updating. Using one step data can't achieve satisfying performance and is a waste of data. Also, reinforcement learning works on sequential condition. Training neural network using one-step data is not beneficial for neural network's generalization.

University of Michigan - Ann Arbor

Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, pooled over many episodes into a replay memory. We then usually sample the memory randomly for a mini-batch of experience, and use this to learn off-policy, as with Deep Q-Networks. 

Experience Replay

Playing Atari with Deep Reinforcement Learning : https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
Human-level control through deep reinforcement learning : https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf

Learn Before

Related