Visual Analysis of Deep Q-network
Dewen Seng, Jiaming Zhang, Xiaoying Shi · KSII Transactions on Internet and Information Systems · 2021
In recent years, deep reinforcement learning (DRL) models are enjoying great interest as their success in a variety of challenging tasks.Deep Q-Network (DQN) is a widely used deep reinforcement learning model, which trains an intelligent agent that executes optimal actions while interacting with an environment.This model is well known for its ability to surpass skilled human players across many Atari 2600 games.Although DQN has achieved excellent performance in practice, there lacks a clear understanding of why the model works.In this paper, we present a visual analytics system for understanding deep Q-network in a non-blind matter.Based on the stored data generated from the training and testing process, four coordinated views are designed to expose the internal execution mechanism of DQN from different perspectives.We report the system performance and demonstrate its effectiveness through two case studies.By using our system, users can learn the relationship between states and Q-values, the function of convolutional layers, the strategies learned by DQN and the rationality of decisions made by the agent.