The 2021/22 course “Bayesian Machine Learning and Information Processing” will start in November 2021 (Q2).
Course goals
This course provides an introduction to Bayesian machine learning and information processing systems. The Bayesian approach affords a unified and consistent treatment of many useful information processing systems.
Course summary
This course covers the fundamentals of a Bayesian (i.e., probabilistic) approach to machine learning and information processing systems. The Bayesian approach allows for a unified and consistent treatment of many model-based machine learning techniques. Initially, we focus on Linear Gaussian systems and will discuss many useful models and applications, including common regression and classification methods, Gaussian mixture models, hidden Markov models and Kalman filters. We will discuss important algorithms for parameter estimation in these models including the Expectation-Maximization (EM) algorithm and Variational Bayes (VB). The Bayesian method also provides tools for comparing the performance of different information processing systems by means of estimating the Bayesian evidence for each model. We will discuss several methods for approximating Bayesian evidence. Next, we will discuss intelligent agents that learn purposeful behavior from interactions with their environment. These agents are used for applications such as self-driving cars or interactive design of virtual and augmented realities. Indeed, in this course we relate synthetic Bayesian intelligent agents to natural intelligent agents such as the brain. You will be challenged to code Bayesian machine learning algorithms yourself and apply them to practical information processing problems.
News and Announcements
-
01-Dec-2021: Last year’s Probabilistic Programming assignments have been made available as exercises. Solutions are given as well.
-
26-Nov-2021: As per the TU/e mandate, there will be no assignment given prior to the Christmas break.
-
13-Nov-2021: This year’s live classes will be online!
-
As much as possible we use the Piazza course site for new announcements.
Instructors
- Prof.dr.ir. Bert de Vries (email: bert.de.vries@tue.nl) is the responsible instructor for this course and teaches all lectures with label B.
- Dr. Wouter Kouw (w.m.kouw@tue.nl) teaches all practical sessions on probabilistic programming with label W.
- Magnus Koudahl, MSc (m.t.koudahl@tue.nl) is the teaching assistant. Mr. Koudahl presents the “What is Life?" bonus lecture.
Materials
In principle, you can download all needed materials from the links below.
Books
Please consider downloading the following books/resources:
- Bert de Vries (2021), PDF bundle of all lecture notes for lessons B0 through B12.
- Wouter Kouw (2021), PDF bundle of all probabilistic programming lecture notes for lessons W1 through W4.
- Christopher M. Bishop (2006), Pattern Recognition and Machine Learning. You can also buy a hardcopy, e.g. at bol.com.
- Ariel Caticha (2012), Entropic Inference and the Foundations of Physics.
Software
- The software installation guide contains step-by-step instructions to setup everything you need to run the course notebooks yourself.
- You can test your installation by running the notebook called Probabilistic-Programming-0.ipynb which can be found in the (github) repo under
lessons/notebooks/probprog.
Lecture notes, videos and exercises
You can access all lecture notes, videos and exercises online through the links below:
| Date | lesson | materials | |||
|---|---|---|---|---|---|
| video guides | lecture notes | exercises | live class | ||
| 17-Nov-2021 | B0: Course Syllabus B1: Machine Learning Overview |
B1 | B0, B1 | B0-B1 | |
| 19-Nov-2021 | B2: Probability Theory Review | B2.1, B2.2 | B2 | B2-ex B2-sol |
B2 |
| 24-Nov-2021 | B3: Bayesian Machine Learning | B3.1, B3.2 | B3 | B3-ex B3-sol |
B3 |
| 26-Nov-2021 | W1: Probabilistic Programming 1 - Intro Bayesian ML | W1.1, W1.2, W1.3 | W1 | W1-ex W1-sol |
W1 |
| 01-Dec-2021 | B4: Factor Graphs and the Sum-Product Algorithm | B4 | B4 | B4-ex B4-sol |
B4 |
| 03-Dec-2021 | B5: Continuous Data and the Gaussian Distribution | B5.1, B5.2, B5.3 | B5 | B5-ex B5-sol |
B5 |
| 08-Dec-2021 | B6: Discrete Data and the Multinomial Distribution | B6 | B6 | B6-ex B6-sol |
|
| 10-Dec-2021 | W2: ProbProg 2 - MP & Analytical Bayesian Solutions | W2.1, W2.2, W2.3 | W2 | W2-ex W2-sol |
|
| 15-Dec-2021 | B7: Regression | B7 | B7 | B7-ex B7-sol |
|
| 17-Dec-2021 | B8: Generative Classification B9: Discriminative Classification |
B8, B9 | B8, B9 | B8-9-ex B8-9-sol |
|
| 22-Dec-2021 | W3: ProbProg 3 - Regression and Classification | W3.1, W3.2 | W3 | W3-ex W3-sol |
|
| 24-Dec-2021 | B10: Latent Variable Models and Variational Bayes | B10 | B10 | B10-ex B10-sol |
|
| break | |||||
| 12-Jan-2022 | B11: Dynamic Models | B11 | B11 | B11-ex B11-sol |
|
| 14-Jan-2022 | B12: Intelligent Agents and Active Inference | B12 | B12 | B12-ex B12-sol |
|
| 19-Jan-2022 | W4: ProbProg 4: Latent Variable and Dynamic Models | W4.1, W4.2 | W4 | W4-ex W4-sol |
|
| 21-Jan-2022 | M1: Bonus Lecture: What is Life? | M1 | |||
| 03-Feb-2022 | written examination (13:30-16:30) | ||||
| 22-Apr-2022 | resit written examination (18:00-21:00) | ||||
- Furthermore, Q&A for each lesson can be accessed at the Piazza course site.
- When you do the exercises, feel free to consult the following matrix and Gaussian cheat sheets (by Sam Roweis) when doing the exercises.
Study Guide
-
Please consult the Course Syllabus (lecture notes for 1st class) for advice on how to study the materials.
-
Each year there will be two written exam opportunities. You cannot bring notes or books to the written exam sessions. All needed formulas are supplied at the exam sheet.
Exam Preparation
In addition to the materials in the above table, we provide two representative practice written exams:
- 2021-01-18: exam A, solutions A; exam B, solutions B
- 2021-04-15: exam, solutions