“Stealing Is Wrong… and Too Hard to Pull Off Successfully”: Inclusion of Normative Versus Capacity Information in Robot Command Rejection
Alyssa Hanson, Gordon Briggs, Ruchen Wen, Yifei Zhu, Tom Williams · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · 2025
In this work, we consider the inclusion of normative vs non-normative information in norm-violation responses. In particular, we consider situations where robots are given requests that simultaneously violate both normative constraints and non-normative capacity constraints. To understand what reasons robots should provide when confronted with such violations, we present the results of a human subjects experiment in which we systematically varied the degrees of both norm and capacity violation in human commands made toward robots, and then measured the effects of this variation on participants’ preferred robot response choices. Our results (1) suggest that robots should consistently include normative information when rejecting a command, and (2) shed light on the contexts in which capacity information should also be included.