News
Important - Regarding Project Phase RegistrationWritten on 11.11.26 by Hevra Petekkaya Hi everyone, As I mentioned in today's tutorial, those who want to participate in the project phase should register for it through CMS latest by 18.11.2026. The registration is already open now, so you can register right away. I will also upload the presentation about the project phase to CMS for… Read more Hi everyone, As I mentioned in today's tutorial, those who want to participate in the project phase should register for it through CMS latest by 18.11.2026. The registration is already open now, so you can register right away. I will also upload the presentation about the project phase to CMS for more details. |
Clarification for the discussion in today's tutorialWritten on 04.11.26 by Hevra Petekkaya Hi everyone, First of all, I've uploaded the PDF version of the slides for today's tutorial, as some of you requested. Second, we had a discussion about two-step flow model sampling and whether we end up at the mean of the target distribution, which was 0 in our example with equally likely… Read more Hi everyone, First of all, I've uploaded the PDF version of the slides for today's tutorial, as some of you requested. Second, we had a discussion about two-step flow model sampling and whether we end up at the mean of the target distribution, which was 0 in our example with equally likely targets (+3 and −3). The answer depends on the step size we take. At (t=0), the model predicts an instantaneous velocity pointing toward the mean of the target distribution. If we take a full Euler step (Delta t=1), we end up exactly at the mean, regardless of our initially sampled noise. However, in two-step sampling, we usually take a step of size (Delta t=0.5). This means multiplying the predicted velocity by 0.5 and adding it to our current position. We therefore only move halfway toward the mean, rather than reaching it, unless our initial noise sample was already 0. We then query the model again at (t=0.5), using our new position (z_{0.5}), to predict an updated velocity and take another step of size 0.5. I hope this clarifies the confusion. If not, you know where and when to find us :) |
Feedback of Tutorial-1Written on 03.11.26 by Hevra Petekkaya Hi everyone, The feedback for Tutorial-1 is completed and uploaded for whoever submitted the assignment. If you have further questions, you can visit mine or Umar's office hours for further clarification.
|
Release of Tutorial 2Written on 30.10.26 by Hevra Petekkaya Hi everyone, The second tutorial is now available, and the submission deadline is November 8. We recommend working on the notebook in Google Colab, which offers free GPU access, as some exercises may run faster on a GPU. Please let us know if you have any questions or run into difficulties. |
Start of TutorialsWritten on 20.10.26 (last change on 20.10.26) by Hevra Petekkaya Hi everyone, This is a kind reminder that we will be starting with our tutorials tomorrow, 21st of October. The slides are uploaded. See you tomorrow! |
Office HoursWritten on 14.10.26 (last change on 14.10.26) by Hevra Petekkaya Below are the office hours for myself, Hevra, and our tutor, Umar. If you have individual questions, you can visit us during the office hours. However, if you have questions which might also concern other students, then please post them in CMS Forum so we won't have to reply individually to common… Read more Below are the office hours for myself, Hevra, and our tutor, Umar. If you have individual questions, you can visit us during the office hours. However, if you have questions which might also concern other students, then please post them in CMS Forum so we won't have to reply individually to common questions.
Office Hours:
Hevra Petekkaya: Thursday, 2 pm - 3 pm (This week, 15th of Oct, not available for the office hour) Location: MPI-INF (E1 4) 6th floor. Room 629
Muhammad Umar Javed - Wednesday, 2 pm - 3 pm Location: MPI-INF (E1 4) 6th floor. Room 611 |
Release of Tutorial 1Written on 13.10.26 by Hevra Petekkaya Tutorial 1 has been released. Please let us know if you face any problems or have questions. Good luck! |
For now, the room will stay the sameWritten on 13.10.26 (last change on 13.10.26) by Jan Eric Lenssen There was the ask if we can switch the lecture room to have tables at the seats. There was the ask if we can switch the lecture room to have tables at the seats. |
Registration is openWritten on 12.10.26 by Jan Eric Lenssen Registration for the course should now be open for the next two weeks (until Oct 26). |
Visual Generative Models
This lecture introduces continuous generative modeling, diffusion and flow matching, as the dominant paradigm for visual synthesis, and traces its extension from static images to dynamic, embodied domains. We start with the core mathematical picture: forward noising processes, score/velocity-field learning, and the shared ODE/SDE view that unifies diffusion models and flow matching as a generative modeling framework that learns a velocity field to transform a source distribution into a target distribution through a continuous flow. From there, we move to video diffusion models and their emerging role as implicit world models, before turning to robotics: Diffusion Policy as a case study in using denoising processes to represent multimodal action distributions, and video generation as a substrate for planning and simulation in physical AI. The lecture closes with open questions around sampling efficiency, controllability, and what it means for a generative model to also be a world model.
Website: https://vip.mpi-inf.mpg.de/vgm.html
CMS Registration will open after the introductory lecture on Oct 12!
(Non-mandatory but recommended) prerequisites
- Linear algebra and multivariate calculus
- Basic probability
- Deep learning fundamentals
- Python and PyTorch
Prerequisites for taking the exam
The second half of the tutorials will include a project phase. Participation is mandatory to be eligible for taking the exam.
- Participation includes
- selecting / defining a project,
- presenting the project idea and motivation in a short presentation in the tutorial,
- presenting the results of the project in a short presentation in the tutorial at the end of the semester.
- All presentations will be during the tutorial hours. There will be available presentation slots over multiple weeks.
Dates and Locations
- Lecture: Mondays, 4pm–6pm (starts at 4 pm, from 2nd lecture), Room E1 5 (MPI-SWS building) R.002
- Tutorial: Wednesdays, 12pm–2pm (starts at 12:15), Room E1 4 (MPI-INF building) R.024
- The first, introductory lecture will be on October 12 and will start at 4:15 pm.
- Tutorials will start on October 21.
Preliminary Schedule of Content
The following is a preliminary, high-level outline of the topics covered and may still change.
Foundations
- Generative modeling as transport: probability paths, flows, and velocity fields
- Flow matching: the conditional flow matching objective
- Diffusion models as a special case of flow matching
- Samplers, discretization error, and the cost of sampling
Practice, Steering and Judgment
- Latents and architectures: autoencoders, tokenizers, diffusion transformers
- Conditioning, guidance and control
- Evaluation: measuring fidelity and coverage, reward models as learned evaluators
Efficiency and Post-Training
- Few-step sampling: distillation, consistency models, flow maps
- Reasoning and RL post-training of flow models
Video
- Video generation: spatiotemporal flows, generation regimes, long-horizon consistency
Embodied Generation and World Models
- Robot learning
- From denoising to velocity-field policies: Diffusion Policy and flow-based policies
- World models and simulators: action conditioning, planning, synthetic data
Material
Online Slides with Animations and Videos
L01, Oct 12 - Introduction: https://vip.mpi-inf.mpg.de/teaching/vgm/l01/
L02, Oct 19 - Transport: https://vip.mpi-inf.mpg.de/teaching/vgm/l02/
L03, Oct 26 - Flow Matching: https://vip.mpi-inf.mpg.de/teaching/vgm/l03/
L04, Nov 02 - Diffusion Models in the Flow Matching Framework: https://vip.mpi-inf.mpg.de/teaching/vgm/l04/
L05, Nov 09 - Sampling: https://vip.mpi-inf.mpg.de/teaching/vgm/l05/
Access info is here.
PDF Versions of the Slides are in Materials Tab
Related External Material
MIT Introduction to Flow Matching and Diffusion Models: https://diffusion.csail.mit.edu/2026/index.html
Flow Matching Cookbook: https://arxiv.org/pdf/2412.06264
Catching Up on Fundamentals
Probability: Seeing Theory, Brown University: interactive and very short. The chapters on basic probability, distributions and Bayesian inference are the relevant ones.
Differential Equations: 3Blue1Brown, Differential equations: the first video covers vector fields and stepping along them, like what our samplers do.
Calculus: 3Blue1Brown, Essence of calculus: derivatives and the chain rule.
Calculus: Khan Academy, Multivariable calculus: the unit on derivatives of multivariable functions covers gradient, divergence and Jacobian.
Neural Networks: PyTorch, Learn the Basics: the official hands-on tutorial, should be enough for the exercises.
Random Cool Visual Generative Model Stuff
Genie 3: https://deepmind.google/models/genie/
Lingbot-World Open Source World Model: https://technology.robbyant.com/lingbot-world
Our Neighbours from Black Forrest: https://bfl.ai/
WorldLabs Atlas: https://www.worldlabs.ai/blog/atlas