Study on High-Level Structure of cognition control construction in Exploration and Exploitation within Multi-Armed Bandit Model of Reinforcement Learning
Jiaxing Tian, Izawa Jun · 2023
Reinforcement learning models have been instrumental in simulating the behaviors of animals and humans, whose actions and choices are largely influenced by rewards. This study proposes a high-level environmental monitoring structure into cognitive models, optimizing the balance between exploration and exploitation in reinforcement learning theory. This approach allows for more flexible adaptation to complex environmental changes. We focus on a multi-armed bandit task, incorporating a hierarchical model that adds a high-level cognition control structure based on the cognitive system. This structure adjusts the balance between exploration and exploitation according to the perceived uncertainty of the environment. In a stable environment, the model minimizes exploration to maximize rewards. In contrast, in an unstable environment, the model promotes rapid exploration to identify the best strategy after environmental changes. This cognition-based hierarchical structure offers valuable insights for future studies about analyzing human responses to environmental uncertainty and exploration behavior.