Relevant Experiences in Replay Buffer
Giang Dao, Minwoo Lee · 2019
The sampling bias problem creates instability in reinforcement learning. Reusing past experiences with experience replay helps reinforcement learning overcome possible sampling bias. After the success of deep reinforcement learning, other variations of experience replay have been proposed and further improved learning performance. However, how to select samples to store in experience replay methods has not been well investigated. In this paper, we examine if there exists relevant (or significant) experiences to be preferably replayed. For systematic discovery of relevant experiences, we adopt the DRL-Monitor and discuss how it improves the efficiency of learning. Comparing with traditional experience replay and prioritized experience replay, we demonstrate improved learning performance in different Atari games.