News
Currently, no news are available
Stochastic Modeling with Markov Chains
Outline
This course aims to provide an introduction on Markov chains in discrete time. The lecture will be delivered in the theorem-proof style for which the final
goals are to present mathematically rigorous proofs for some classical results in the topic of Markov chains together with their applications.
The main content includes:
- Markov chains; Transition probabilities; Chapman-Kolmogorov equations;
- Classification of states: Communicating and absorbing states; Recurrence and transience;
- Hitting probabilities and mean hitting times; Birth and death chains, in particular M/M/1 model;
- 1/2/3-dimensional random walks;
- Stationary distribution; Convergence to the equilibrium; Ergodic theorem;
- PageRank Algorithm and its probabilistic proof;
- Markov Chain Monte Carlo methods; Metropolis-Hasting algorithm; Gibbs sampler.
References
- R. Durrett, Essentials of Stochastic Processes, 3rd eds., Springer, 2016. (available online in the IP range of Saarland University)
- O. Häggström, Finite Markov Chains and Algorithmic Applications, Cambridge, 2002. (available online in the IP range of Saarland University)
- J. Norris, Markov Chains, Cambridge, 1997.
Prerequisites
To be able to follow the course, you need to know some basic concepts in probability/analysis/linear algebra such as: conditional probability, law of total probability, distribution of a discrete random variable, convergence/divergence of series of real numbers, eigenvalues and eigenvectors of a square matrix,
etc.
For example, a (solid) knowledge of courses "Mathematics for computer science 1, 2, 3" would be sufficient to follow.
Registration
All further information and course material can be found on the learning management system Moodle. If you are interested to attend the course, please ask for the enrolment key at your earliest convenience via email to:
The course will be held in English.
Lecture
Wednesday 14:15 - 15:45 Starting date: 12 April 2023 Place: Building E2 4, room 1.15, HS IV.
Tutorial
Wednesday 9:00 - 10:00 am Starting date: 19 April 2023 Place: Building E2 5, Seminarraum 1 (U.37).