From MDP to AlphaZero
David SeWell · 2021
In this paper I will explain the AlphaGo family of algorithms starting from first principles and requiring little previous knowledge from the reader. The focus will be upon one of the more recent versions AlphaZero but I hope to explain the core principles that allowed these algorithms to be so successful. I will generally refer to AlphaZero as theses core set of principles and will make it clear when I am referring to a specific algorithm of the AlphaGo family. AlphaZero in short combines Monte Carlo Tree Search (MCTS) with Deep learning and self-play. We will see how these three concepts fit together and we will break down each of these pieces and look at examples to clarify understanding. I implemented a simplified version of the algorithm on TicTacToe and Connect4 and the code is available online as well as a simple web app that allows you to play against a trained agent. https://github.com/befeltingu/VikingZero https://github.com/befeltingu/VikingDashboard