An Evaluation of Low-Shot Learning Techniques for the Detection of Credit Card Fraud
Preston Billion-Polak, Taghi M. Khoshgoftaar · 2024
The task of credit card fraud detection presents many obstacles to traditional machine learning techniques, most notably severe class imbalance. Meanwhile, Low-Shot Learning (LSL) techniques have shown great effectiveness in multi-class problems involving significant class rarity, but have seen little examination in binary tasks such as fraud detection. In this paper, we select two low-shot learning papers from the literature (representing two major approaches to LSL), and replicate their methods on a highly imbalanced credit-card fraud dataset, to compare their performances to that of six contemporary state-of-the-art (SOTA) models. In the process, we improve on the experimental methodologies presented in the two selected papers by introducing multiple runs of cross-fold validation, measurement of the imbalance-robust performance metric Area Under the Precision-Recall Curve (AUPRC), and statistical analyses of our results. To the best of our knowledge, our work is the first to compare the two different LSL approaches to non-LSL SOTA methods, as well as the first peer-reviewed paper to evaluate any LSL method on a fraud detection dataset. We find that, while our chosen optimization-based method, Meta-Balance, underperforms the SOTA baseline in terms of AUPRC, our similarity-based method, Siamese-RNN, significantly outperforms the SOTA, and yields the highest AUPRC score recorded in the literature on the Kaggle credit card fraud dataset.