Fictitious Self-Play in Extensive-Form Games
Johannes Heinrich, David Lawrence Silver, Google DeepMind · 2015
Fictitious play is a popular game-theoretic model of learning in games. However, it has received little attention in practical applications to large problems. This paper introduces two variants of fictitious play that are implemented in be-havioural strategies of an extensive-form game. The first variant is a full-width process that is re-alization equivalent to its normal-form counter-part and therefore inherits its convergence guar-antees. However, its computational requirements are linear in time and space rather than exponen-tial. The second variant, Fictitious Self-Play, is a machine learning framework that implements fictitious play in a sample-based fashion. Ex-periments in imperfect-information poker games compare our approaches and demonstrate their convergence to approximate Nash equilibria. 1.