Influence Based Fitness Shaping for Coevolutionary Agents
Everardo Gonzalez, Sandhya Viswanathan, Kagan Tumer · Proceedings of the Genetic and Evolutionary Computation Conference · 2024
Coevolving cooperative teams creates a challenging joint-action discovery problem because fitness functions generally evaluate team performance rather than individual agent performance. Feedback "sparsity" where agents only receive feedback when they jointly stumble upon a valuable action compounds this problem. Fitness shaping techniques alleviate this problem by extracting agent contributions and providing stepping stone incentives in sparse feedback settings. However, such techniques require agents to make direct and measurable impacts to system performance. If agents have indirect impacts, such as influencing other agents to accomplish tasks, existing shaping methods fail to provide adequate feedback. In this work, we introduce Influence Based Fitness Shaping (IBFS) to capture and incentivize indirect impacts. IBFS extracts an agent's impact based on how it influences other agents and guides exploration towards influencing actions in sparse feedback settings. Our results in a multiagent shepherding problem show that IBFS outperforms standard fitness shaping, and the gains increase when feedback becomes sparser.