Multi-Task Shared Transformer in Image Anomaly Detection of Fusion Learning Framework
Anran Wu, Wen An · 2025
Detecting anomalous images is highly important for different applications including quality inspection in industrial domains, and analysis of medical images. Traditional approaches tend to focus on single tasks, which do not adequately address the breadth and complexity of anomalous characteristics. To address this issue, we propose a Fusion Learning Framework (FL-MST) based on Multi-task Shared Transformer (MST), which incorporates three types of sub-tasks: reconstruction detection, classification judgement, and semantic localization, as part of a unified architecture or system. FL-MST, based on the improved Masked Autoencoder and Data-efficient Image Transformer (DeiT), implements a shared Transformer encoder to extract general features and task-specific decoders to model target differentiation. To strengthen the interaction and information flow between the different tasks, Cross-Task Attention Fusion Module (CTAFM) is implemented, along with multi-view contrast loss and task adaptive optimization to enhance robustness. Experiments have validated the effectiveness of FL-MST capable of outperforming several existing and dominant approaches on both image-level discrimination and pixel-level localization.