Is Your Game Generator Working? Evaluating Gemini, an Intentional Generator
Joseph Osborn, Melanie Dickinson, Barrett R. Anderson, Adam Summerville, Jill Denner, David Torres, Noah Wardrip–Fruin, Michael Mateas · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · 2019
Determining whether a game generator is working properly is challenging, since it entails conducting potentially many evaluations of generated games and synthesizing these into a net evaluation of the system. The problem is compounded when the generator has a human-centered goal: for example, that the generated games should be interpreted as having certain mechanics or as being about particular ideas. In this paper, we examine the Gemini game generator and develop an evaluation instrument that tests the interpretability of its generated games’ mechanics and higher-order proceduralist arguments. In the process we build empirical evidence for the claim that some amount of non-systems-based framing is required in order for arguments made by procedural rhetorics to be sensible to players. The tools we have assembled for this evaluation can be applied to game generators more broadly; game generators should be allowed to invent games which go beyond merely formally “good” or subjectively “fun.”