Reproducibility Analysis of Recommender Systems relying on Visual Features: traps, pitfalls, and countermeasures
Pasquale Lops, Elio Musacchio, Cataldo Musto, Marco Polignano, Antonio Silletti, Giovanni Maria Semeraro · 2023
Reproducibility is an important requirement for scientific progress, and the lack of reproducibility for a large amount of published research can hinder the progress over the state-of-the-art. This concerns several research areas, and recommender systems are witnessing the same reproducibility crisis. Even solid works published at prestigious venues might not be reproducible for several reasons: data might not be public, source code for recommendation algorithms might not be available or well documented, and evaluation metrics might be computed using parameters not explicitly provided. In addition, recommendation pipelines are becoming increasingly complex due to the use of deep neural architectures or representations for multimodal side information involving text, images, audio, or video. This makes the reproducibility of experiments even more challenging.