Bayesian Deep Reinforcement Learning via Deep Kernel Learning
Junyu Xuan, Jie Lü, Zheng Yan, Guangquan Zhang · International Journal of Computational Intelligence Systems · 2018
Reinforcement learning (RL) aims to resolve the sequential decision-making under uncertainty problem where an agent needs to interact with an unknown environment with the expectation of optimising the cumulative long-term reward.Many real-world problems could benefit from RL, e.g., industrial robotics, medical treatment, and trade execution.As a representative model-free RL algorithm, deep Q-network (DQN) has recently achieved great success on RL problems and even exceed the human performance through introducing deep neural networks.However, such classical deep neural network-based models cannot well handle the uncertainty in sequential decision-making and then limit their learning performance.In this paper, we propose a new model-free RL algorithm based on a Bayesian deep model.To be specific, deep kernel learning (i.e., a Gaussian process with deep kernel) is adopted to learn the hidden complex action-value function instead of classical deep learning models, which could encode more uncertainty and fully take advantage of the replay memory.The comparative experiments on standard RL testing platform, i.e., OpenAI-Gym, show that the proposed algorithm outweighs the DQN.Further investigations will be directed to applying RL for supporting dynamic decision-making in complex environments.