Causal XAI

Causal XAI

This seminar explores the intersection of causality, explainability, and interpretability in modern machine learning. While traditional methods often rely on statistical associations to understand model behavior, causal approaches aim to uncover the underlying cause-and-effect mechanisms that drive predictions and decision-making. The seminar explores the promises and challenges of causality in Explainable AI (XAI) through the following thematic blocks: 

 

Block I: Counterfactual explanations and Causal Interventions

Block II: Attribution Methods and Causality

Block III: Concept Bottlenecks and Causality

Block IV: Representation learning and causality

 

Students will be divided into four teams (one team per seminar block). Each team will work collaboratively throughout the seminar, guided by the instructors, to conduct a literature review on their assigned topic. The final goal of each team is to prepare and deliver a tutorial that introduces the topic, explains its evolution over time, and discusses the current state of the art.

 

Key sources include:

• Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (3rd ed.)

• Limitations of Interpretable Machine Learning Methods

---

Topics

Block 1 – Counterfactuals and Interventions

Students will explore techniques for counterfactual reasoning and causal interventions, including counterfactual explanations, causal recourse, and feature-effect visualization methods. They will learn why ignoring causality can be misleading and how causal models enable intervention-consistent and actionable recommendations.

Keywords: counterfactual explanations, causal recourse, causal interventions, Partial Dependence Plots (PDPs), Ceteris Paribus Plots.

Sources - Seminal work: 

• Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (3rd ed.)

• Limitations of Interpretable Machine Learning Methods

• Counterfactual explanations without opening the black box: Automated decisions and the GDPR.

• On the relationship between explanation and prediction: A causal view.

• Algorithmic recourse: from counterfactual explanations to interventions. 

---

Block 2 – Attribution methods and Causality

This block introduces fundamental post-hoc explainability methods, discusses their limitations, and interprets these limitations in causal terms. It further explores how causal reasoning can improve attribution methods.

Key keywords: feature importance, variable importance under hidden confounding, SHAP values, causality

Supplementary topics: Machine Learning - Additive Models (GAMs), Influential instances, influence functions, prototypes and criticisms, local surrogate models (LIME) 

Sources - Seminal work: 

• Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (3rd ed.)

• Limitations of Interpretable Machine Learning Methods

• A unified approach to interpreting model predictions.

• https://shap.readthedocs.io/en/latest/ , or any other of the plethora of sources for SHAP.

• https://scikit-learn.org/stable/modules/permutation_importance.html 

• Causal shapley values: Exploiting causal knowledge to explain individual predictions of complex models.

• Causal SHAP: Feature Attribution with Dependency Awareness through Causal Discovery.

• Doctor Rashomon and the UNIVERSE of Madness: Variable Importance with Unobserved Confounding and the Rashomon Effect.

---

Block 3 – Concept Bottlenecks and Causality

Introduce concept-based neural networks, such as concept bottleneck models, present their limitations and see how to introduce causality into them to alleviate their problems.

Keywords: concept bottleneck models, concept-based explanations, causal CBMs

Sources - Seminal work: 

• Concept bottleneck models

• Causally reliable concept bottleneck models

• Towards reasonable concept bottleneck models.

• Counterfactual concept bottleneck models.

---

Block 4 – Representation learning and Causality

How to move beyond statistical correlations toward representations that capture the underlying (interpretable) causal structure of the data.

Keywords: causal representation learning, spurious correlations, interventions, causal concept-Based Representation Learning

Sources - Seminal work: 

• Representation learning: A review and new perspectives.
• Toward causal representation learning.

• Beta-VAE: Learning basic visual concepts with a constrained variational framework

• From causal to concept-based representation learning.

---

Requirements

This seminar is primarily aimed at master's students in Computer Science, Data Science and Artificial Intelligence (DSAI), or related fields, who have prior knowledge of machine learning. Participants are expected to have completed one or more courses related to machine learning, such as Core Machine Learning, Elements of Machine Learning, Linear Algebra, Neural Networks, or equivalent courses. 

Prior exposure to topics in explainability, causality, or trustworthy AI is beneficial but not strictly required. In particular, the seminars Explainable Machine Learning, Advanced Topics in Causality and Causal Machine Learning Research, Causethical ML and Trustworthy Machine Learning provide valuable background and will help students engage more deeply with the seminar material. 

Time and place

Tuesdays, 12:00 to 14:00

Room: 2.06 at E1.1

 

Attendance

The seminar will be held in person, and attendance is mandatory. 

 

General structure

The students will be divided into four teams (one team per seminar block), with three students per team. Each team will work collaboratively throughout the seminar, guided by the instructors, to conduct a literature review on their assigned topic. The final goal is for each group to prepare and deliver a tutorial that introduces the topic, explains how it has evolved over time, and discusses the current state of the art.

The seminar will include three types of sessions:

• Background sessions: Sessions led by the instructors to provide the necessary foundations in representation learning and guidance on how to approach and understand research papers.

• Private group meetings: Informal meetings with each team to discuss their progress, evaluate the current state of their presentation, address questions, identify missing elements or unclear aspects, and provide feedback on the content, structure, and their overall understanding of the research landscape.

• Public presentations: Sessions where each team presents their tutorial to the rest of the class, followed by questions and discussion.

Background sessions (3 sessions)

The seminar will start with three background sessions designed to give students the necessary tools to begin their literature review.

Building a tutorial I: Foundations, Definitions, and Research Questions (4 sessions)

The goal of this stage is for each team to build a strong understanding of the foundations of their topic. Their presentation should introduce the key definitions and concepts, present the main approaches in the literature, and discuss the limitations and open questions that motivated subsequent research.

• Private group meetings (2 sessions)

  • One private meeting per team (45 min).

• Public presentation (2 sessions)

  • One presentation per team (45 min for presentation + Q&A).

Building a Tutorial II: Evolution of the Topic and Current State of the Art (4 sessions)

During this stage, students should expand their presentation to cover the evolution of the field, including major developments, influential works, recent advances, and current open research directions and challenges. They should also incorporate their own perspective and draw their own conclusions. At this stage, each team is expected to discuss at least 9 papers (a minimum of 3 papers per team member).

• Private group meetings (2 sessions)

  • One private meeting per team (45 minutes).

• Public presentation (2 sessions)

  • One presentation per team (45 min for presentation + Q&A)

Tentative Calendar

Date	Session Type	Content
October 13	Background	Introduction to Causality
October 20	Background	Introduction to Explainability
October 27	Background	How to read a paper?
2 weeks without seminar sessions
November 17	Private meetings (1, 2)	Building a tutorial I
November 24	Private meetings (3, 4)	Building a tutorial I
1 week without seminar sessions
December 8	Public presentations (1, 2)	Building a tutorial I
December 15	Public presentations (3, 4)	Building a tutorial I
3 weeks without seminar sessions
January 12	Private meetings (1, 2)	Building a tutorial II
January 19	Private meetings (3, 4)	Building a tutorial II
1 week without seminar sessions
February 2	Public presentations (1, 2)	Building a tutorial II
February 9	Public presentations (3, 4)	Building a tutorial II

 

* Each team has 2 weeks between the private group meeting and the presentation, and 4 weeks between their first presentation and the next private meeting.

Selecting topic

We will share a form with the students to indicate their preferences.

• Release form to indicate preferences: October 13 (first background session)

• Release team assignment: October 20 (second background session, 3 weeks before the first private group meeting)

 

Deliverables and grading scheme

TBA