Stochastic Modeling with Markov Chains Thuan Nguyen


Currently, no news are available

Stochastic Modeling with Markov Chains


This course aims to provide an introduction on Markov chains in discrete time. The lecture will be delivered in the theorem-proof style for which the final
goals are to present mathematically rigorous proofs for some classical results in the topic of Markov chains together with their applications.

The main content includes:

  • Markov chains; Transition probabilities; Chapman-Kolmogorov equations;
  • Classification of states: Communicating and absorbing states; Recurrence and transience;
  • Hitting probabilities and mean hitting times; Birth and death chains, in particular M/M/1 model;
  • 1/2/3-dimensional random walks;
  • Stationary distribution; Convergence to the equilibrium; Ergodic theorem;
  • PageRank Algorithm and its probabilistic proof;
  • Markov Chain Monte Carlo methods; Metropolis-Hasting algorithm; Gibbs sampler.



To be able to follow the course, you need to know some basic concepts in probability/analysis/linear algebra such as: conditional probability, law of total probability, distribution of a discrete random variable, convergence/divergence of series of real numbers, eigenvalues and eigenvectors of a square matrix,
For example, a (solid) knowledge of courses "Mathematics for computer science 1, 2, 3" would be sufficient to follow.


All further information and course material can be found on the learning management system Moodle. If you are interested to attend the course, please ask for the enrolment key at your earliest convenience via email to:

 nguyen [at]

The course will be held in English.

Moodle enrolment


Wednesday 14:15 - 15:45     Starting date: 12 April 2023     Place: Building E2 4, room 1.15, HS IV.


Wednesday 9:00 - 10:00 am     Starting date: 19 April 2023     Place: Building E2 5, Seminarraum 1 (U.37).

Privacy Policy | Legal Notice
If you encounter technical problems, please contact the administrators