News
Exam & Sheet 6Written on 26.07.23 by Johannes Hoffmann The date for the exam has been fixed. All information regarding the exam can be found on the main page. The full solution to sheet 6 will be available soon. There was a problem with ReLU in exercise 3, where the scaling factor of 1/sqrt(m) should not have been part of the network function, but… Read more The date for the exam has been fixed. All information regarding the exam can be found on the main page. The full solution to sheet 6 will be available soon. There was a problem with ReLU in exercise 3, where the scaling factor of 1/sqrt(m) should not have been part of the network function, but instead part of the initialization of the vector a. The exercise will be updated accordingly. Apologies for the inconvenience. 
No more lecturesWritten on 10.07.23 by Roland Speicher

Lecture 20  incomplete videoWritten on 06.07.23 by Roland Speicher The video of Lecture 20 has been uploaded. Note, however, that due to technical problems, the last 20 minutes of the lecture were not recorded. 
Exercise sheet 6Written on 03.07.23 by Roland Speicher The new (and last) assignment has been uploaded. 
UpdatesWritten on 27.06.23 by Johannes Hoffmann

ExamWritten on 23.06.23 by Johannes Hoffmann There will be a written exam for the High Dimensional Analysis. After Sheet 6 is graded we will know who is admitted to the exam, so we will then try to find a date for the exam that suits everyone that is interested in participating. 
Sheet 3Written on 01.06.23 by Johannes Hoffmann The feedback for sheet 3 should now be available. Also, please refer to the bottom of the main page for updates regarding the status of the exercise classes. 
UpdatesWritten on 30.05.23 by Johannes Hoffmann Exercise sheet 4 has been released. The date for the exercise class for sheet 3 has been changed from tomorrow to one week later (June 7, usual time slot). Since nobody showed up for the last exercise class, future exercise classes will only take place if at least one participant requests this in… Read more Exercise sheet 4 has been released. The date for the exercise class for sheet 3 has been changed from tomorrow to one week later (June 7, usual time slot). Since nobody showed up for the last exercise class, future exercise classes will only take place if at least one participant requests this in advance (just send me an email). The feedback for sheet 3 will be available in the next few days. 
New exercise sheet & tutorialsWritten on 01.05.23 by Johannes Hoffmann Exercise sheet 2 is now available. The first tutorial will be on Wednesday, May 3, in SR 2 (E2.5) at 14:15. 
Tutorial time slotWritten on 27.04.23 by Johannes Hoffmann Please indicate your preference for the tutorial time slot via Doodle. The slot with the most votes wins. The deadline is defined as "when I look at the results on Monday, May 1", so try to vote Sunday at the latest. 
Updates to sheet 1Written on 23.04.23 by Johannes Hoffmann Two changes were made to sheet 1: first, in exercise 4(c), mu and sigma were missing their normalization constants, which have been added. Second, the estimate in exercise 4(b) was replaced with one that works better with the task. Please excuse the inconvenience. 
Schedule for next weekWritten on 20.04.23 by Johannes Hoffmann There will be no regular lectures next week. Instead you should watch an earlier video of Roland Speicher on the Wick Formula (up to 1:16:33). This explains and proves the Wick formula (which is a combinatorial formula for moments of Gaussian variables and of products of independent Gaussian… Read more There will be no regular lectures next week. Instead you should watch an earlier video of Roland Speicher on the Wick Formula (up to 1:16:33). This explains and proves the Wick formula (which is a combinatorial formula for moments of Gaussian variables and of products of independent Gaussian variables). This formula will be used later in our course; the main facts about the Wick formula will be provided then, but the video offers more details. Additionally, instead of the lecture on Wednesday, April 26, there will be a Q&A session for the first exercise sheet. As May 1 is a holiday, the next regular lecture will be on Wednesday, May 3. 
First exercise sheet availableWritten on 17.04.23 by Johannes Hoffmann The first exercise sheet is now available under Materials. Submissions can be handed in via CMS starting from 22.04.2023. Please compile your submission into one pdf. Up until the evening of the 21st, you can create or join teams to hand in the exercises in groups of up to three people (or you can… Read more The first exercise sheet is now available under Materials. Submissions can be handed in via CMS starting from 22.04.2023. Please compile your submission into one pdf. Up until the evening of the 21st, you can create or join teams to hand in the exercises in groups of up to three people (or you can hand them in as a singleton, then you don't have to create a team). 
Starting time of the lecturesWritten on 16.04.23 by Roland Speicher All lectures will start c.t., i.e. at 10:15 a.m. 
First LectureWritten on 05.04.23 by Roland Speicher The first lecture will be on Wednesday, April 12, at 10:15. 
RegistrationWritten on 14.03.23 (last change on 14.03.23) by Roland Speicher Registration is now open for the course.

High Dimensional Analysis: Random Matrices and Machine Learning
4h Topics Course (9 CP)
Analysis in highdimensional spaces is at the basis of an understanding and rigorous treatment of dealing with large data sets. In a very rough sense one has to understand functions of many variables, in particular also their generic and statistical behaviour.
Highdimensional spaces are actually quite strange and exotic, when considered from our experience in small dimensions (remember that we live in three dimensions). On one side, highdimensional spaces are VERY large, so it is easy to get lost there (a phenomenon often known under the name “curse of dimensionality”), but on the other hand, functions of many variables often concentrate to constant values and thus allow very precise answers to apparently intractable questions (a phenomenon going under the name of “concentration phenomenon” and also considered as the “blessing of dimensionality”).
Also statistics in high dimensions with many variables and many observations is different from what one is used to in a classical setting. In classical statistics the number of variables is small compared to the number of observations and essentially one can understand the distribution of the variables via the law of large numbers by doing many measurements. In modern statistics the number of variables is of the same order as the number of observations, so the relation between the real and the measured properties of the variables is more tricky and new mathematical tools are needed to unravel this. In particular, random matrices (which were originally introduced by the statistician Wishart) are such a tool.
The neural networks of modern deep learning are in some sense a special class of functions of many variables, built out of (random) matrices and also some entrywise nonlinear functions. Thus random matrices should help to say something about neural nets, but the latter also present new and interesting extensions of random matrices.
In this course we will take a trip through high dimensions. We hope not to get lost, but to get a feeling for its strange and beautiful behaviour. Much of the terrain is still unexplored, but there are some first interesting islands which have been discovered, waiting for us to explore them.
This course is intended for everybody with an interest in a mathematical rigorous discussion of highdimensional phenomena. A basic mathematical background on the level of the MfI 13 courses should be sufficient (but also necessary). Be aware, however, our intention is not to develop better and faster algorithms for deep learning, but to touch upon some mathematical theories which might (or might not, who knows) be important for a sound mathematical theory of deep learning.
Tentative List of Topics
 curse and blessing of dimensionality
 concentration of vectors and matrices in high dimensions
 Wishart matrices and MarchenkoPastur law
 signalplusnoise models
 neural networks, overparameterization, neural tangent kernel, feature learning and all that
Bibliography
Here are some references on which the lectures will partly rely or which might provide some further reading. In any case they should give an idea what the course will be about. We will not follow one source exclusively, but pick the best from each of them, according to the taste and interest of the lecturer.
 R. Couillet and Z. Liao: Random Matrix Methods for Machine Learning, Cambridge University Press, 2022
 D.A. Roberts and S. Yaida: The Principles of Deep Learning Theory, Cambridge University Press, 2022
 R. Vershynin: HighDimensional Probability, Cambridge University Press, 2018
 M.J. Wainwright: HighDimensional Statistics, Cambridge University Press, 2019
 A. Zagidullina: HighDimensional Covariance Matrix Estimation, Springer, 2021
 the Tensor programs framework of Greg Yang
 A. S. Bandeira, A. Singer, T. Strohmer: Mathematics of Data Science (draft)
This list is not yet final and will increase in due time.
See also the Semesterapparat/Course Reference Shelf provided by the Campus Library for Computer Science and Mathematics.
Time and Place
Lectures: Mondays and Wednesdays, 1012 in HS IV, building E 2.4. The lectures start c.t., i.e., at 10:15 a.m.
Tutorials: Wednesdays, 1416 in SR 2, building E2.5. The tutorials also start c.t., i.e., at 14:15.
 Exercise class 1: May 3
 Exercise class 2: May 17 (canceled)
 Exercise class 3: June 7
 Exercise class 4: June 14
 Exercise class 5: June 28 (canceled)
 Exercise class 6: July 19 (canceled)
Exam
The exam will be on Wednesday, August 9, in lecture hall III (HS III), from 14:15 to 16:15.
You are allowed to bring your own notes to the exam, as long as they are on sheets of paper (handwritten or printed is both fine). The content can be anything you like, but keep it to a reasonable length/amount and don't bring any books. In particular, any use of an electronic device (calculator, phone, laptop, ...) during the exam will be considered cheating.
Don't forget to register via HISPOS/LSF!
Exam inspection
The exam inspection will be on Friday, August 11, in lecture hall IV (HS IV) in building E2.4, from 13:30 to 14:00.