Redefining the Behavior Space for Multi-Objective MAP-Elites
Anna Nickelson, Kagan Tumer · Proceedings of the Genetic and Evolutionary Computation Conference Companion · 2024
Autonomous agents are increasingly deployed in complex real-world environments where they must balance multiple goals and adapt to novel scenarios. Unfortunately, learning on the fly to handle new situations can produce dangerous or unstable policies. Creating a diverse set of pre-trained policies overcomes this problem, and evolving those policies using multiple objectives increases their applicability in many domains. Multi-Objective Map Elites (MOME) addresses this issue by searching for a population of policies that are both high performing on a set of objectives as well as diverse across a set of defined behavior metrics, referred to as the Behavior Descriptor (BD). However, the characteristics of the BD that enhance learning and improve search across the behavior and multiobjective spaces in an Reinforcement Learning (RL) setting are still relatively unknown. This work investigates qualities of the BD for MOME that improve search for good policies, in turn, enabling better decision-making. We show that redefining the behavior space can improve coverage of the multi-objective space by up to 36%.