Adaptation in Constant Utility Non-Stationary Environments.
Michael L. Littman, David H. Ackley · 1991
Environments that vary over time present a fundamental problem to adaptive systems. Although in the worst case there is no hope of effective adaptation, some forms environmental variability do provide adaptive opportunities. We consider a broad class of non-stationary environments, those which combine a variable result function with an invariant utility function, and demonstrate via simulation that an adaptive strategy employing both evolution and learning can tolerate a much higher rate of environmental variation than an evolution-only strategy. We suggest that in many cases where stability has previously been assumed, the constant utility non-stationary environment may in fact be a more powerful viewpoint. 1 Non-stationary environments An adaptive system within an environment performs two basic tasks. First, there is the search for, and the representation of, regularities in the history of interactions with the environment. Second, there is the attempt to gain some advantage from th...