Harnessing Disagreement to Create AI-Powered Systems That Reflect Our Values
Mitchell Gordon · 2021
How do we build artificial intelligence systems that reflect our values? Competing potential values we may want to choose from are, at their core, made up of disagreements between individual people. But while the raw datasets that most ML systems rely on are made up of individuals, today’s approaches to building AI typically abstract the individuals out of the pipeline. My thesis contributes a set of algorithms and interactive systems that re-imagine the pipeline for designing and evaluating AI systems, requiring that we deal with competing values in an informed and intentional way. I start from the insight that at each stage of the pipeline, we need to treat individuals as the key unit of operation, rather than the abstractions or aggregated pseudo-humans in use by today’s approaches. I instantiate this insight to address two problems that arise from today’s AI pipeline. First, evaluation metrics produce actively misleading scores about the extent to which some people’s values are being reflected. I introduce a mathematical transformation that more closely aligns metrics with the values and methods of user-facing performance measures. Second, the resulting AI systems either surreptitiously choose which values to listen to without input from users, or simply present several different outcomes to users without mechanisms to help them select an outcome grounded in their values. I propose a new interaction paradigm for deploying classifiers, asking users to compose a jury consisting of the individual people they’d like their classifiers’ decisions to come from.