Policy shaping with human teachers
Thomas Cederborg, Ishaan Grover, Charles Lee Isbell, Andrea L. Thomaz · 2015
In this work we evaluate the performance of a pol-icy shaping algorithm using 26 human teachers. We examine if the algorithm is suitable for human-generated data on two different boards in a pac-man domain, comparing performance to an oracle that provides critique based on one known winning pol-icy. Perhaps surprisingly, we show that the data generated by our 26 participants yields even bet-ter performance for the agent than data generated by the oracle. This might be because humans do not discourage exploring multiple winning policies. Additionally, we evaluate the impact of different verbal instructions, and different interpretations of silence, finding that the usefulness of data is af-fected both by what instructions is given to teach-ers, and how the data is interpreted. 1