Performance evaluation of interactive evolutionary computation with tournament-style evaluation
Hiroshi Takenouchi, Masataka Tokumaru, Noriaki Muranaka · 2012
We describe the effectiveness of interactive evolutionary computation (IEC) with tournament-style evaluation to reduce the evaluation load of IEC users. Most previous studies did not clearly demonstrate the effectiveness of the tournament-style evaluation. Therefore, we implemented a tournament-style evaluation for a specific application and inspected the effectiveness of the IEC with tournament-style evaluation using an experiment with real users. We used three evaluation objects: music, animation, and image. We evaluated the performance of the following three methods. The first was a normal IGA (NIGA), which is a conventional 10-stage evaluation. The second was a tournament-style evaluation with two levels (T2), which evaluates only the superiority or inferiority of two candidates at a time. The third was a tournament-style evaluation with four levels (T4), which progressively evaluates the superiority or inferiority of two candidates. We inspected the effectiveness of these methods by simulation using an evaluation agent that imitated human preferences (or Kansei). The simulation results showed that the evolution performances of the NIGA and T2 are higher than those of the T4. Also, we inspected the effectiveness of these methods by an evaluation experiment with 42 subjects in their 20s. The experiment results showed that the satisfaction level for generated candidates were approximately equal among the NIGA, T2, and T4. Moreover, with the T2, it was easiest for test subjects to evaluate solution candidates than with the NIGA in all evaluation objects. And with the T4, it was easier for test subjects to evaluate solution candidates when the evaluation objects were music and image than with the NIGA.