Metamorphic Robustness Testing for Recommender Systems: A Case Study
Chengying Mao, YI Xiao-rong, Tsong Yueh Chen · 2020
Recommender system has been widely used in many Internet-based applications such as e-commerce and content provision. In the existing studies, the focus is mainly on its accuracy, while its robustness and trustworthiness are often ignored. Targeting at the “non-testable” problem of recommender systems, this paper proposes a solution of applying metamorphic testing to validate their robustness. According to the characteristics of most prediction programs in recommender systems, five types of metamorphic relations are proposed. Meanwhile, taking a well-known recommender library LibRec as an example, empirical experiments are conducted to verify the proposed solution and metamorphic relations. The results show that they are effective in revealing the robustness problem of recommender systems, and metamorphic testing can also be regarded as a quality evaluation approach for recommendation algorithms or models.