Causal XAI

• Explain, not just describe: Show that you understand the intuition behind concepts and methods.

• Be critical: Discuss assumptions, limitations, and unresolved problems rather than simply summarizing papers.

• Connect ideas: Build a clear narrative from the initial approaches to later developments and remaining challenges.

• Support your arguments: Use concrete examples and evidence from the literature.

• Demonstrate ownership: Be prepared to defend your choices, explain your assigned papers, and answer questions.

Detailed structure

Background sessions (2+1 sessions)

The seminar will start with two background sessions designed to give students the necessary tools to begin their literature review. An additional, optional session on how to read academic papers will be offered by our chair on October 29, from 12:15 to 13:45, in Room 0.01, Building E1.7. The paper [A Simple Framework for Contrastive Learning of Visual Representations] will be used as an example. Please note that this room is shared with another seminar, which takes priority. As a result, seating is limited, and availability cannot be guaranteed.

Building a tutorial I: Foundations, Definitions, and Research Questions (4 sessions)

The goal of this stage is to establish a strong understanding of the foundations of the assigned topic. Students should introduce the key concepts, explain the foundational approaches in the literature, and identify the limitations and open questions that motivated subsequent research.

Private group meetings (2 sessions)

• One private meeting per team (45 min).

• Two teams per session.

• Deliverables and expectations:

  • Briefly walk the instructors through the slides planned for the first public presentation, covering the motivation, key concepts, foundational approaches, and research gaps. This is not a rehearsal! The goal is to review the slides and explain what the team plans to present and why.

  • Explain the material clearly and demonstrate an understanding of the underlying concepts and methods.

  • Be prepared to answer questions and discuss the limitations and assumptions of the approaches presented.

  • Use the feedback received to improve the content and structure of the public presentation.

Public presentation (2 sessions)

• One presentation per team (30 min for presentation + 15 min for Q&A).

• Two presentations per session.

• The presentation should cover:

  • Motivation and problem framing: Why the topic matters, its applications, and the problem it addresses.

  • Key definitions and concepts: Correct explanations, supported by intuition and examples.

  • Foundational approaches: The main initial methods, their core ideas, and how they differ.

  • Limitations and research gaps: The assumptions and limitations of the foundational approaches, and the open questions they leave unresolved.
     

Building a Tutorial II: Evolution of the Topic and Current State of the Art (4 sessions)

Building on Tutorial I, students should expand their presentation to explain how the field has evolved, how subsequent research has addressed the initial gaps, and what remains unresolved. The presentation should cover major developments, influential works, recent advances, and current research challenges. Teams should also develop their own perspective and draw reasoned conclusions. Tutorial II should not feel like a second, longer version of Tutorial I. While Tutorial I establishes the foundations and identifies research gaps, Tutorial II explains how the field has responded to those gaps and critically assesses its current state.

Each team is expected to discuss at least 9 papers (a minimum of 3 papers per team member), with possible exceptions for specific seminar blocks.

Private group meetings (2 sessions)

• One private meeting per team (45 minutes).

• Deliverables and expectations:

  • Present the planned slides for Tutorial II, including a revised introduction incorporating feedback from Tutorial I.

  • Discuss the selected papers and explain how they relate to the research gaps identified in Tutorial I.

  • Explain the contributions, methods, assumptions, results, and limitations of the selected papers.

  • Identify remaining open questions and develop a critical perspective on the literature.

  • Be prepared to discuss the material in depth and answer questions.

Public presentation (2 sessions)

• One presentation per team (35 min for presentation + 10 min for Q&A)

• The presentation should cover

  • Recap: A concise review of the essential foundations and research gaps from Tutorial I.

  • Evolution of the field: A coherent storyline showing how the literature developed and how papers build on, differ from, or improve upon earlier work.

  • Analysis of individual papers: The problem addressed, core method, intuition, assumptions, key results, and limitations of each paper.

  • Remaining challenges: Specific open questions that follow from the literature.

  • Own perspective and conclusions: A reasoned assessment of what has been achieved, what remains unresolved, and which future directions are promising.

Topics

Block 1 – Counterfactuals and Interventions

Students will explore techniques for counterfactual reasoning and causal interventions, including counterfactual explanations, causal recourse, and feature-effect visualization methods. They will learn why ignoring causality can be misleading and how causal models enable intervention-consistent and actionable recommendations.

Main instructor: Rickey

Keywords: counterfactual explanations, causal recourse, causal interventions, Partial Dependence Plots (PDPs), Ceteris Paribus Plots.

Sources - Seminal work: 

• Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (3rd ed.)

• Limitations of Interpretable Machine Learning Methods

• Counterfactual explanations without opening the black box: Automated decisions and the GDPR.

• On the relationship between explanation and prediction: A causal view.

• Algorithmic recourse: from counterfactual explanations to interventions. 

Block 2 – Attribution methods and Causality

This block introduces fundamental post-hoc explainability methods, discusses their limitations, and interprets these limitations in causal terms. It further explores how causal reasoning can improve attribution methods.

Main instructor: Nektarios

Keywords: feature importance, variable importance under hidden confounding, SHAP values, causality

Supplementary topics: Machine Learning - Additive Models (GAMs), Influential instances, influence functions, prototypes and criticisms, local surrogate models (LIME) 

Sources - Seminal work: 

• Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (3rd ed.)

• Limitations of Interpretable Machine Learning Methods

• A unified approach to interpreting model predictions.

• https://shap.readthedocs.io/en/latest/ , or any other of the plethora of sources for SHAP.

• https://scikit-learn.org/stable/modules/permutation_importance.html 

• Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models.

• Causal SHAP: Feature Attribution with Dependency Awareness through Causal Discovery.

• THE FAILURES OF MARGINAL INFLUENCE-BASED ATTRIBUTION METHODS FOR GLOBAL TIME SERIES EXPLANATIONS

• Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect.

Block 3 – Concept Bottlenecks and Causality

Introduce concept-based neural networks, such as concept bottleneck models, present their limitations and see how to introduce causality into them to alleviate their problems.

Main instructor: Nektarios

Keywords: concept bottleneck models, concept-based explanations, causal CBMs

Sources - Seminal work: 

• Concept bottleneck models

• Causally reliable concept bottleneck models

• Towards reasonable concept bottleneck models.

• Counterfactual concept bottleneck models.

Block 4 – Representation learning and Causality

How to move beyond statistical correlations toward representations that capture the underlying (interpretable) causal structure of the data.

Main instructor: Rickey
Keywords: causal representation learning, spurious correlations, interventions, causal concept-Based Representation Learning

Sources - Seminal work: 

• Representation learning: A review and new perspectives.
• Toward causal representation learning.

• Beta-VAE: Learning basic visual concepts with a constrained variational framework

• From causal to concept-based representation learning.