3D and 4D Computer Vision

3D and 4D Computer Vision

---

Course Description

Computer Vision is an interdisciplinary research field at the intersection of machine learning and signal processing. It studies techniques for automatic analysis and interpretation of visual and multi-modal input data. This lecture focuses on one of the advanced subfields of Computer Vision related to the inference of the observed higher-dimensional structures (3D and deformable 3D) from lower-dimensional observations (2D images).

Our world is inherently non-rigid at different spatial and temporal scales. Reconstructing and modelling it in 4D from visual observations is a vibrant research field that remains challenging and that has numerous practical applications, for instance, in AR/VR/XR, computer game development, human-computer interaction and sport analytics. The frequent challenges of this field include the ill-posedness of the underlying optimisation problems and settings (e.g., monocular, which is of special interest in Computer Vision) and observed scene conditions (e.g., partial observations, low light or high-speed motions), among many others. The goal of the lecture "3D and 4D Computer Vision" is to introduce foundational concepts of 3D computer vision for deformable and composite scenes (4D = 3D + time) as well as results of the latest research in the field in a systematic and structured manner through generalisation of studied concepts from 3D to 4D cases.

The lecture will cover the fundamentals of 3D computer vision applicable across a wide range of 3D and 4D settings (multiple view geometry, triangulation, stereo vision, bundle adjustment, linear transformations, parametrisations of rotations), different types of visual sensors (RGB, event and depth cameras), 3D and 4D scene representations, deformation models and regularisers, non-rigid structure from motion (NRSfM), shape-from-template, correspondence problems, novel-view synthesis of non-rigid scenes, generative and diffusion models in 4D vision, 3D human pose estimation, egocentric 4D vision as well as video generation of composite scenes. Apart from milestone methods in the field, the lecture will discuss several recent works on 4D vision including state-of-the-art approaches. This lecture is accompanied by triweekly theoretical exercises.

 

Schedule and Other Details

Course webpage: https://4dqv.mpi-inf.mpg.de/teaching/3D_and_4D_Computer_Vision/

Workload: 6 Credit Points

Format: Weekly lectures + triweekly exercises

Time: Wednesdays: 14:15 – 15:45 p.m. (lecture) / Thursdays: 14:15 – 15:45 p.m. (exercise)

Location: Wednesdays: 002 in E1 5 / Thursdays: 024 in E 1 4

Exam: Oral (exam period: 16.02.2026–20.03.2026). 

Exam admission: At least 50% of correctly accomplished exercises.

 

Prerequisites

The lecture "3D and 4D Computer Vision" targets students in the following programs: Visual Computing (M.Sc.), Computer Science (M.Sc.), and Data Science and Artificial Intelligence (M.Sc.).

Some background in linear algebra, calculus, image processing, computer graphics and machine learning is recommended.

 

Course Content

15.10.25: Format, Overview and Introduction
22.10.25: Fundamentals of 3D and 4D Computer Vision I
29.10.25: Fundamentals of 3D and 4D Computer Vision II
05.11.25: 3D and 4D Scene Representations
12.11.25: 3D Correspondence Problems
19.11.25: Multi-View Geometry, Structure from Motion and Bundle Adjustment
26.11.25: 3D Volumetric Rendering of Rigid and Non-rigid Scenes
03.12.25: General Monocular and Depth-Based 4D Reconstruction
10.12.25: 3D Human Pose Estimation
17.12.25: No lecture
07.01.26: Egocentric 4D Vision and Human Pose Estimation
14.01.26: Generative and Diffusion Models in 4D Vision
21.01.26: Controllable Video Generation
28.01.26: Event-based Vision I
04.02.26: Event-based Vision II
11.02.26: Recap and Recent Research in 4D Computer Vision

 

Exercise Sheets and Tutorials

23.10.25: Exercise sheet 1 "Fundamentals of 3D and 4D Computer Vision" released
30.10.25: Questions (exercise sheet 1)
06.11.25: Tutorial 1
13.11.25: Exercise sheet 2 "Scene Representations and Correspondence Problems" released
20.11.25: Questions (exercise sheet 2)
27.11.25: Tutorial 2
04.12.25: Exercise sheet 3 "Structure from Motion, Bundle Adjustment, 3D Volumetric Rendering and Monocular 4D Reconstruction" released
11.12.25: Questions (exercise sheet 3)
18.12.25: Tutorial 3
08.01.26: Exercise sheet 3 "3D Human Pose Estimation" released
15.01.26: Questions (exercise sheet 4)
22.01.26: Tutorial 4
29.01.26: Exercise sheet 5 "Generative and Event-based Vision" released
05.02.26: Questions (exercise sheet 5)
12.02.26: Tutorial 5

---