Credit assignment method for learning effective stochastic policies in uncertain domains
Sachiyo Arai, Katia P. Sycara · 2001
In this paper, weintroduce FirstVisit ProfitSharing (FVPS) as a credit assignment procedure, an important issue in classifier systems and reinforcement learning frameworks. FVPS reinforces effective rules to makean agent acquire stochastic policies that cause it to behavevery robustly within uncertain domains, without pre-defined knowledge or subgoals. We use an internal episodic memory, not only to identify perceptual aliasing states but also to discard looping behavior and to acquire effectivestochastic policies to escape perceptual deceptive states. We demonstrate the effectiveness of our method in some typical classes of Partially Observable Markov Decision Processes, comparing with Sarsa() using a replacing eligibility trace. We claim that this approach results in an effective stochastic or deterministic policy which is appropriate for the environment. 1