News
Kickoff Meeting: Friday, 17 AprilWritten on 14.04.26 by Kamila Szewczyk Dear Students, We have scheduled the kick-off meeting for Friday, 17th of April. The meeting will take place in E2.1 SR106. No preparation is necessary. We will go over the seminar modalities and a brief introduction to the topic. Please be there on time, at 12:00 (sharp!). |
Proseminar Introduction to Information Theory
In this proseminar, we will cover topics from two books on information theory and compression:
- Thomas M. Cover & Joy A. Thomas, "Elements of Information Theory".
- David J. C. MacKay, "Information Theory, Inference, and Learning Algorithms".
Registration Rules
You cannot register directly here.
If you were assigned to this proseminar via the seminar assignment system, you should have received an email with a token to register in the CMS.
List of Topics
- Information & Entropy.
- AEP & typical sets.
- Entropy rates; Markov chains; HMMs.
- Symbol codes: Kraft, Huffman.
- Arithmetic coding.
- Universal coding; Lempel Ziv.
- Statistical modelling: PPM, CTW, BWT.
- Gambling, Kelly, side information.
- Information theory & the stock market.
- Shannon secrecy & the wiretap channel.
- Channel capacity; noisy-channel theorem.
- Basic noisy coding: Hamming, RM, cyclic.
- BCH, Reed/Solomon & list decoding.
- Rate/distortion theory.
- Network information theory.
- Continuous channels; LDPC & FC.
Workload and Grading
The workload for the proseminar [5 CP] is 150 hours over the semester, including preparation, reading, meetings, preparing a talk; providing feedback.
Further organizational information will be discussed at the kickoff meeting.