From data to decision: a multi-stage framework for class imbalance mitigation in optical network failure analysis
Yousuf Moiz Ali, Jaroslaw E. Prilepsky, Nicola Sambo, João Pedro, Mohammad M. Hosseini, Antonio Napoli, Sergei Konstantinovich Turitsyn, Pedro J. Freire · Journal of Optical Communications and Networking · 2025
Machine learning-based failure management in optical networks has gained significant attention in recent years, but severe class imbalance, where normal instances far outnumber failure cases, remains a considerable challenge. While pre- and in-processing techniques have been widely studied, post-processing methods are largely unexplored. We present a direct comparison of pre-, in-, and post-processing approaches for class imbalance mitigation in failure detection and identification using an experimental dataset. For failure detection, post-processing, particularly threshold adjustment, yields the highest F1 score improvement of up to 15.3%, while random under-sampling offers the fastest inference. In failure identification, generative AI methods deliver the most significant performance gains up to 24.2%, whereas post-processing has a limited impact in multi-class settings. When class overlap exists and latency is critical, over-sampling methods like synthetic minority over-sampling technique (SMOTE) are most effective; without latency constraints, meta-learning excels. In low-overlap scenarios, generative AI approaches provide the best performance with minimal inference time.