Enhanced framework for generating counterfactual images with sophisticated caption and inversion-free image editing
Li Xiang, Ren Togo, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama · 2025
The training process of classification models is commonly based on real-world data and models may learn certain spurious relationships, leading to an over-reliance on features not directly relevant to the subject. In this paper, we propose a novel framework for generating counterfactual images. Our framework enables us to confirm whether classification models are sensitive to changes in the features under consideration. We have introduced the latest caption and image generator, which enables better counterfactual image generation as well as more efficient processing. Experimental results show that the counterfactual images generated by our method have superior feature perturbation capabilities, which allows us to assess the robustness of the classification model more effectively. The main improvements our framework offers over existing methods are the generation of higher-quality counterfactual images and the reduction of the computational cost of this process.