The Value Learning Problem
Nate Soares · 2018
Autonomous AI systems&s; programmed goals can easily fall short of programmers&s; intentions. Even a machine intelligent enough to understand its designers&s; intentions would not necessarily act as intended. We discuss early ideas on how one might design smarter-than-human AI systems that can inductively learn what to value from labeled training data, and highlight questions about the construction of systems that model and act upon their operators&s; preferences.