Near-Duplicate Build Failure Detection from Continuous Integration Logs
M H Li, Mika Mäntylä, Jesse Nyyssölä, Matti Luukkainen · 2025
Addressing build failures is important in software development, particularly within Continuous Integration and Deployment (CI/CD) pipelines. A common issue arises when a developer encounters a new build failure that has occurred in the past. We refer to such build failures as near-duplicate build failures. We collect build logs from a GitHub Actions CI/CD pipeline. We label near-duplicate build failures by identifying those that have the same failed tests. We propose a framework for detecting near-duplicate build failures through log similarity analysis. As the majority of log lines in failed builds are similar to passing logs, we propose using an Out-of-Vocabulary Detector (OOVD) filtering to identify only failure-relevant lines and to improve near-duplicate detection accuracy. Our results suggest a noticeable improvement using OOVD filtering across all metrics for Top-K results (K = 5, precision@K: 0.864 vs. 0.526 and MAP@K: 0.941 vs. 0.814). Finding near-duplicate build failures can be an important software engineering challenge that, to the best of our knowledge, has not been studied in the past.