Dean Bodenham and Niall Adams’s contribution to the Discussion of ‘the Discussion Meeting on Probabilistic and statistical aspects of machine learning’
Dean A. Bodenham, Niall M. Adams · Journal of the Royal Statistical Society Series B (Statistical Methodology) · 2023
We congratulate the authors for their thought-provoking paper and innovative approach for detecting changepoints in a supervised manner. The results in Table 1 and Figure 2 show that the proposed approach has good performance in terms of the misclassification error rate metric, or in other words in determining whether or not a sequence contains a changepoint. However, since a primary concern in changepoint detection is the accuracy of the localization of the changepoint, it would be interesting to see the performance of the proposed approach for offline changepoint detection metrics such as the covering metric (Arbelaez et al., 2010), where the locations of the detected changepoints are compared with the locations of the true changepoints (van den Burg & Williams, 2020). It would also be instructive to see this performance in comparison to established offline methods such as the pruned exact linear time method (Killick et al., 2012) or the wild binary segmentation method (Fryzlewicz, 2014), rather than the online cumulative sum method. Another concern is how the proposed method would handle very long sequences, for example, of length more than one million. Besides computational challenges, one might expect most methods to flag a changepoint in such sequences, while the exact location of the changepoint(s) may be more difficult to determine. Finally, we suggest that one potential area of application for the proposed method may be in cellular biology where cells are tracked and their protein expression levels are recorded. These time series are often short in length, potentially N≤20, which can be a challenging scenario for traditional, unsupervised changepoint detection methods, although for this application it may be possible to obtain labelled examples from experts for the necessary training. We look forward to future developments of the proposed approach.