Setting Up ML Agents Toolkit
Abhilash Majumder · Apress eBooks · 2020
We have seen in the previous chapters how states, actions, and rewards play a crucial role in driving an agent to reach its goal in a reinforcement learning (RL) environment. Now that we are familiar with the fundamentals of generic state-based RL, we will move ahead with the installation of all the libraries, frameworks, and extensions related to the usage of Unity ML agents as well as to further progress in the later modules of deep learning. Before we dive into the installation, let us try to understand the ML agents package made by Unity. Since its inception in 2017, the Unity ML Agents Toolkit has provided researchers, developers, and game programmers with a plethora of resources in the field of deep RL. The ML Agents Toolkit was initially created with the vision to assist researchers and developers in transforming games and simulations created with the help of Unity Editor into deep learning environments where the agents can be trained using state-of-the-art (SOTA) deep RL algorithms, evolutionary and genetic strategies, and other deep learning methods (involving computer vision, synthetic data generation) through a simplified Python API. With the latest release of package version 1.0 (at the time of writing this book), there have been huge advancements in the development of the Toolkit in terms of streamlining the C# SDK, Python API linking, robust compatibility with Tensorflow library, and huge advancements in areas of curriculum learning, self- and adversarial play, generative adversarial imitation learning, and pre-existing deep learning algorithms (PPO, SAC). There have been significant modifications of the core library that use the open AI Gym environment as a wrapper. The Unity ML Agents Toolkit version 1.0 has stable communicators linking Unity (C#) and the Python API (deep learning) and has provided game developers with the flexibility of writing deep learning agents and simulating their own AI in their games. Researchers, who are interested in modifying the core algorithms for their use-cases, are also provided with a flexible Gym wrapper environment, which provides a template for writing their own deep learning algorithms. All of these can be done with the simple usage of the Unity Engine, Unity ML Toolkit, Jupyter Notebook, and Tensorflow framework. Now that we understand the scope and possibilities of Unity ML Agents, we will discuss the entire process of installing all the necessary libraries and frameworks in this chapter. The different agents in the Unity ML Agents Toolkit are presented in Figure 3-1.