Inverse Norm Conflict Resolution
Daniel Kasenberg, Matthias J. Scheutz · 2018
In previous work we provided a "norm conflict resolution" algorithm allowing agents in stochastic domains (represented by Markov Decision Processes) to "maximally satisfy" a set of moral or social norms, where such norms are represented by statements in linear temporal logic (LTL). This required the agent designer to provide weights specifying the relative importance of each norm. In this paper, we propose an "inverse norm conflict resolution'' algorithm for learning these weights from demonstration. This approach minimizes a cost function based on the relative entropy between a policy encoding the observed behavior and a policy representing optimal norm-following behavior. We demonstrate the effectiveness of the algorithm in a simple GridWorld domain.