Learning Nonlinear Activation Functions in RL Through Evolutionary Computation
Coen Nusse, Jacob E. Kooi · 2025
This study explores the development of nonlinear activation functions in reinforcement learning through evolutionary computation. Traditional activation functions like Rectified Linear-Unit (ReLU) and hyperbolic tangent (tanh) face limitations such as vanishing gradients and poor adaptability across varied reinforcement learning environments. By leveraging evolutionary algorithms within the JAX framework, we evolve activation functions that dynamically adjust to task complexities in environments such as MinAtar (Breakout, SpaceInvaders) and Mujoco (Hopper, Ant). Using Proximal Policy Optimization (PPO) as a foundational algorithm, the research demonstrates that meta-learned activation functions can improve learning efficiency and agent performance compared to conventional functions. However, experiments reveal that no single optimal activation function emerges consistently; each evolutionary run produces distinct solutions tailored to specific tasks. This finding emphasizes the context-dependent nature of evolved activation functions, highlighting the adaptability of the approach but also the challenge of generalization across environments. The results suggest potential advancements for creating more flexible, efficient learning models in reinforcement learning.