Testing the Robustness of AutoML Systems
Tuomas Halvari, Jukka K. Nurminen, Tommi Mikkonen · Electronic Proceedings in Theoretical Computer Science · 2020
Automated machine learning (AutoML) systems aim at finding the best machine learning (ML) pipeline that automatically matches the task and data at hand. We investigate the robustness of machine learning pipelines generated with three AutoML systems, TPOT, H2O, and AutoKeras. In particular, we study the influence of dirty data on accuracy, and consider how using dirty training data may help create more robust solutions. Furthermore, we also analyze how the structure of the generated pipelines differs in different cases.