Agent's Feedback in Preference Elicitation
Miroslav Kárný, Tereza Siváková · 2021
A generic decision-making (DM) agent specifies its preferences partially. The studied prescriptive DM theory, called fully probabilistic design (FPD) of decision strategies, has recently addressed this obstacle in a new way. The found preference completion and quantification exploits that:$\blacktriangleright$FPD models the closed DM loop and the agent's preferences by joint probability densities (pds); there is$a$preference-elicitation (PE) principle, which maps the agent's model of the state transitions and its incompletely expressed wishes on an ideal pd quantifying them. The gained algorithmic quantification provides ambitious but potentially reachable DM aims. It suppresses demands on the agent selecting the preference-expressing inputs. The remaining PE options are:$\blacktriangleright$a parameter balancing exploration with exploitation;$\blacktriangleright$a fine specification of the ideal (desired) sets of states and actions;$\blacktriangleright$relative importance of these ideal sets. The current paper makes decisive steps towards a systematic and realistic choice of such inputs by solving a meta-DM task. The algorithmic “meta-agent” observes the user's satisfaction, expressed by school-type marks, and tunes the free PE inputs to improve these marks. The solution requires a suitable formalisation of such a meta-task. This is done here. The proposed way copes with the danger of infinite regress and the dimensionality curse. Non-trivial simulations illustrate the results.