MuAE: A Mutation Testing Framework for Evaluating Autoencoders
Samer Y. Khamaiseh, Steven Chiacchira, Anas Alsobeh, Aibak Aljadayah · 2025
While autoencoders are pivotal in critical applications such as anomaly detection and medical imaging, their reliability remains understudied compared to supervised models. Although mutation testing has advanced for neural networks, no framework exists for assessing autoencoder robustness against real-world faults, leaving a gap in safety-critical validation. Furthermore, autoencoders lack explicit labels, rendering traditional mutation metrics ineffective. We propose MuAE, the first mutation testing tool tailored for autoencoders, addressing this gap through: (1) a fault taxonomy derived from real-world debugging cases; (2) mutation operators that inject faults while preserving model validity; and (3) reconstruction error (Erec) as an evaluation metric to quantify fault impacts on output fidelity. We validate MuAE on CIFAR-10 using two convolutional autoencoders and four mutation operators (M1, M2, M3), generating 3 mutants per operator. Mutations significantly degraded performance: layer reinitialization increased Erecby up to 210%, while weight noise caused milder degradation. This condensed evaluation demonstrates the feasibility of mutation-based robustness assessment for unsupervised models and highlights potential links to adversarial vulnerability.