Component-Based Evaluation Using GLMM.
Nicola Ferro, Gianmaria Silvello · Padua Research Archive (University of Padova) · 2017
Topic variance has a greater effect on performances than system variance but it cannot be controlled by system developers who can only try to cope with it.On the other hand, system variance is important on its own, since it is what system developers may affect directly by changing system components and it determines the differences among systems.In this paper, we face the problem of studying system variance in order to better understand how much system components contribute to overall performances.To this end, we propose a methodology based on General Linear Mixed Model (GLMM) to develop statistical models able to isolate system variance, component effects as well as their interaction.This is an extended abstract of [1].Please refer to the original paper for the full model and experimental results.