Experience Replay

Experience Replay is a technique in Reinforcement Learning (RL) where an agent stores its past transitions (state, action, reward, next state) in a buffer, then trains on random mini-batches sampled from that buffer.

Sampling randomly breaks the correlation between consecutive experiences, and lets the agent learn from each experience many times, which makes training more stable and data-efficient.

It was a key ingredient of Deep-Q Learning, described in Playing Atari with Deep Reinforcement Learning.