Offline and online evaluation of news recommender systems at swissinfo.ch
Florent Garcin, Boi V. Faltings, Olivier Donatsch, Ayar Alazzawi, Christophe Bruttin, Amr Huber · 2014
We report on the live evaluation of various news recommender systems conducted on the website swissinfo.ch. We demonstrate that there is a major difference between offline and online accuracy evaluations. In an offline setting, recommending most popular stories is the best strategy, while in a live environment this strategy is the poorest. For online setting, context-tree recommender systems which profile the users in real-time improve the click-through rate by up to 35%. The visit length also increases by a factor of 2.5. Our experience holds important lessons for the evaluation of recommender systems with offline data as well as for the use of the click-through rate as a performance indicator.