PUB - Publikationen an der Universität Bielefeld

There is a growing interest in methods that explain predictions of image classification models to increase algorithmic transparency. Counterfactual Explanations (CFEs) provide a causal explanation as they introduce changes in the original image that change the classifier’s prediction. Current counterfactual generation approaches suffer from the fact that they potentially modify a too large region in the image that is not entirely causally related to a classifier’s decision, thus not always providing targeted explanations. We propose a new method, Attention Based Counterfactuals via CycleGAN (ABC-GAN), that combines attention-guided object translation with counterfactual image generation via Generative Adversarial Networks. To generate an explanation, ABC-GAN incorporates both a counterfactual loss and the classifier’s attention mechanism. By leveraging the attention map generated by GradCAM++, ABC-GAN alters regions in the image that are important for the classifier’s prediction. This approach ensures that the generated explanation focuses on the specific areas that contribute to the change in prediction while preserving the background and non-salient regions of the original image. We apply our approach to medical X-ray datasets (MURA Bone X-Ray, RSNA Chest X-ray) and compare it to state-of-the-art methods. We demonstrate the feasibility and, in the case of the MURA dataset, the superiority of ABC-GAN in all the measured metrics with the highest percentage of counterfactuals (99% Validity) and image similarity. On the other dataset, our method outperforms the competitive methods in small changes and image similarity. We argue that ABC-GAN is thus beneficial for classification problems requiring precise and minimal CFEs.