Opponent Exploitation Based on Bayesian Strategy Inference and Policy Tracking
Kuei-Tso Lee, Yen-Yun Huang, Je-Ruei Yang, Sheng‐Jyh Wang · IEEE Transactions on Games · 2023
In a multiagent competitive environment, it is important for an agent to detect the opponent's policy and adopt a suitable policy to exploit the opponent. Conventionally, most methods, e.g., Bayesian policy reuse (BPR) variants, assume that the opponent adopts a fixed policy or a randomly changing policy. In this article, we make a more realistic and reasonable assumption that the opponent may select its policy based on the previous observation. Here, we define the term “strategy” as the mapping from the previous observation to the opponent's selected policy, and we propose the Bayesian strategy inference (BSI) framework to infer the opponent's strategy. Furthermore, to deal with opponents who may randomly select their policies, the BSI framework is combined with an intraepisode policy tracking mechanism to construct the Bayesian strategy inference plus policy tracking (BSI-PT) algorithm. In our experiments, we design anExtended Batter versus Pitchergame (EBvPG) for the evaluation of the proposed BSI-PT framework. The experimental results demonstrate that BSI-PT obtains higher policy prediction accuracy and winning percentage than three other BPR variants against the opponents with a specific policy selection strategy, with a random selection strategy, or with a partially random strategy.