Evaluation of Moving Window Policies with CART
Sousuke Amasaki, Chris Lokan · 2016
CONTEXT: Recent studies have shown that estimation accuracy can be affected by only using a window of recent projects (instead of all past projects) as training data for building an effort estimation model. The effect and its extent can be affected by the effort estimation methods used, and the windowing policy used (fixed size or fixed duration). The generality of the windowing approach remains uncertain, because only a few effort estimation methods have been examined with each policy. OBJECTIVE: To investigate the effect on estimation accuracy of using the fixed-duration window policy, particularly in comparison to the fixed-size window policy, when using Classification and Regression Trees (CART) as the estimation method. METHOD: Using a single-company ISBSG data set studied previously in similar research, we examine the effects of using a fixed-duration windowing policy on the accuracy of estimates using CART. RESULTS: Fixed-duration windows rarely improve the accuracy of estimates with CART, compared to using all past projects as training data. Few window sizes lead to statistically significant differences. The effect is smaller than when fixed-size windows are used. CONCLUSIONS: Fixed-duration windows are not helpful with this data set when using CART as the estimation method. The results support the preference for the fixed-size window policy that was found in previous research. This contributes further to understanding the effect of using windows.