Disentangling User and Item Sequence Patterns in Sequential Recommendation Data Sets
K. Liu, Yang Liu, Alan J. Medlar, Dorota Głowacka · 2025
Sequential recommenders use the ordering of user-item interactions to perform next-item prediction. Several studies have attempted to estimate how much sequential information is available in data sets used for the offline evaluation of sequential recommenders by randomly shuffling users' interaction histories and breaking the sequential dependencies between interactions. However, random shuffling fails to distinguish between sequential patterns from user behaviour (users consuming items based on previous interactions) and item availability (when items enter the system and become available for user consumption).In this article, we analyse several widely used data sets in sequential recommendation studies using two shuffling techniques: random shuffling and constrained shuffling. While random shuffling reorders interactions arbitrarily, constrained shuffling does not allow user-item interactions to occur prior to the item's first appearance in the data set. Our experiments show that sequential information can either come exclusively from user behaviour patterns or item availability, or from a combination of the two. These findings have implications for understanding evaluation results in sequential recommendation and highlights why some data sets may be less appropriate for offline evaluation given how little sequential information comes from user behaviour.