Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation
Vaeth P, M. Fruehwald A, Paaßen B, Gregorova M (2023)
In: Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML). Guidotti R, Naretto F, Cerquitelli T, Biecek P, Regoli D (Eds); .
Konferenzbeitrag
| Veröffentlicht | Englisch
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Autor*in
Vaeth, Philipp;
M. Fruehwald, Alexander;
Paaßen, BenjaminUniBi 
;
Gregorova, Magda
;
Gregorova, MagdaHerausgeber*in
Guidotti, Riccardo;
Naretto, Francesca;
Cerquitelli, Tania;
Biecek, Przemyslaw;
Regoli, Daniele
Abstract / Bemerkung
Latest methods for visual counterfactual explanations (VCE) harness the power of deep generative models to synthesize new examples of high-dimensional images of impressive quality. However, it is currently difficult to compare the performance of these VCE methods as the evaluation procedures largely vary and often boil down to visual inspection of individual examples and small scale user studies. In this work, we propose a framework for systematic, quantitative evaluation of the VCE methods and a minimal set of metrics to be used. We use this framework to explore the effects of certain crucial design choices in the latest diffusion-based generative models for VCEs of natural image classification (ImageNet). We conduct a battery of ablation-like experiments, generating thousands of VCEs for a suite of classifiers of various complexity, accuracy and robustness. Our findings suggest multiple directions for future advancements and improvements of VCE methods. By sharing our methodology and our approach to tackle the computational challenges of such a study on a limited hardware setup (including the complete code base), we offer a valuable guidance for researchers in the field fostering consistency and transparency in the assessment of counterfactual explanations.
Erscheinungsjahr
2023
Titel des Konferenzbandes
Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML)
Konferenz
5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML)
Konferenzort
Turin, Italy
Konferenzdatum
2023-09-18 – 2023-09-18
Page URI
https://pub.uni-bielefeld.de/record/2990550
Zitieren
Vaeth P, M. Fruehwald A, Paaßen B, Gregorova M. Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation. In: Guidotti R, Naretto F, Cerquitelli T, Biecek P, Regoli D, eds. Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML). 2023.
Vaeth, P., M. Fruehwald, A., Paaßen, B., & Gregorova, M. (2023). Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation. In R. Guidotti, F. Naretto, T. Cerquitelli, P. Biecek, & D. Regoli (Eds.), Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML)
Vaeth, Philipp, M. Fruehwald, Alexander, Paaßen, Benjamin, and Gregorova, Magda. 2023. “Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation”. In Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML), ed. Riccardo Guidotti, Francesca Naretto, Tania Cerquitelli, Przemyslaw Biecek, and Daniele Regoli.
Vaeth, P., M. Fruehwald, A., Paaßen, B., and Gregorova, M. (2023). “Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation” in Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML), Guidotti, R., Naretto, F., Cerquitelli, T., Biecek, P., and Regoli, D. eds.
Vaeth, P., et al., 2023. Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation. In R. Guidotti, et al., eds. Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML).
P. Vaeth, et al., “Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation”, Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML), R. Guidotti, et al., eds., 2023.
Vaeth, P., M. Fruehwald, A., Paaßen, B., Gregorova, M.: Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation. In: Guidotti, R., Naretto, F., Cerquitelli, T., Biecek, P., and Regoli, D. (eds.) Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML). (2023).
Vaeth, Philipp, M. Fruehwald, Alexander, Paaßen, Benjamin, and Gregorova, Magda. “Diffusion-based Visual Counterfactual Explanations – Towards Systematic Quantitative Evaluation”. Proceedings of the 5th International Workshop on eXplainable Knowledge Discovery in Data Mining (XKDD) at the European Conference on Machine Learning (ECML). Ed. Riccardo Guidotti, Francesca Naretto, Tania Cerquitelli, Przemyslaw Biecek, and Daniele Regoli. 2023.
Open Access
arXiv: 
Google Scholar