Bayesian Machine Learning and Information Processing (5SSD0)

academic year 2021/22

The 2021/22 course “Bayesian Machine Learning and Information Processing” will start in November 2021 (Q2).

Course goals

This course provides an introduction to Bayesian machine learning and information processing systems. The Bayesian approach affords a unified and consistent treatment of many useful information processing systems.

Course summary

This course covers the fundamentals of a Bayesian (i.e., probabilistic) approach to machine learning and information processing systems. The Bayesian approach allows for a unified and consistent treatment of many model-based machine learning techniques. Initially, we focus on Linear Gaussian systems and will discuss many useful models and applications, including common regression and classification methods, Gaussian mixture models, hidden Markov models and Kalman filters. We will discuss important algorithms for parameter estimation in these models including the Expectation-Maximization (EM) algorithm and Variational Bayes (VB). The Bayesian method also provides tools for comparing the performance of different information processing systems by means of estimating the Bayesian evidence for each model. We will discuss several methods for approximating Bayesian evidence. Next, we will discuss intelligent agents that learn purposeful behavior from interactions with their environment. These agents are used for applications such as self-driving cars or interactive design of virtual and augmented realities. Indeed, in this course we relate synthetic Bayesian intelligent agents to natural intelligent agents such as the brain. You will be challenged to code Bayesian machine learning algorithms yourself and apply them to practical information processing problems.

News and Announcements

  • 28-Jan-2022: The Probabilistic Programming assignments have been graded (solution notebook available from Assignment section).

  • 22-Dec-2021: The Probabilistic Programming assignment has been made available (see Assignment section below) ahead of schedule.

  • 01-Dec-2021: Last year’s Probabilistic Programming assignments have been made available as exercises. Solutions are given as well.

  • 26-Nov-2021: As per the TU/e mandate, there will be no assignment given prior to the Christmas break.

  • 13-Nov-2021: This year’s live classes will be online!

  • As much as possible we use the Piazza course site for new announcements.

Instructors

Materials

In principle, you can download all needed materials from the links below.

Books

Please consider downloading the following books/resources:

Software

Lecture notes, videos and exercises

You can access all lecture notes, videos and exercises online through the links below:

Date lesson materials
video guides lecture notes exercises live class
17-Nov-2021 B0: Course Syllabus
B1: Machine Learning Overview		 B1 B0, B1 B0-B1
19-Nov-2021 B2: Probability Theory Review B2.1, B2.2 B2 B2-ex
B2-sol		 B2
24-Nov-2021 B3: Bayesian Machine Learning B3.1, B3.2 B3 B3-ex
B3-sol		 B3
26-Nov-2021 W1: Probabilistic Programming 1 - Intro Bayesian ML W1.1, W1.2, W1.3 W1 W1-ex
W1-sol		 W1
01-Dec-2021 B4: Factor Graphs and the Sum-Product Algorithm B4 B4 B4-ex
B4-sol		 B4
03-Dec-2021 B5: Continuous Data and the Gaussian Distribution B5.1, B5.2, B5.3 B5 B5-ex
B5-sol		 B5
08-Dec-2021 B6: Discrete Data and the Multinomial Distribution B6 B6 B6-ex
B6-sol		 B6
10-Dec-2021 W2: ProbProg 2 - MP & Analytical Bayesian Solutions W2.1, W2.2, W2.3 W2 W2-ex
W2-sol		 W2
15-Dec-2021 B7: Regression B7 B7 B7-ex
B7-sol		 B7
catch-up
17-Dec-2021 B8: Generative Classification
B9: Discriminative Classification 		B8, B9 B8, B9 B8-9-ex
B8-9-sol		 B8-9
22-Dec-2021 B10: Latent Variable Models and Variational Bayes B10 B10 B10-ex
B10-sol		 B10
break
10-Jan-2022 W3: ProbProg 3 - Regression and Classification W3.1, W3.2 W3 W3-ex
W3-sol		 W3
12-Jan-2022 B11: Dynamic Models B11 B11 B11-ex
B11-sol		 B11
14-Jan-2022 B12: Intelligent Agents and Active Inference B12 B12 B12-ex
B12-sol		 B12
19-Jan-2022 W4: ProbProg 4: Latent Variable and Dynamic Models W4.1, W4.2 W4 W4-ex
W4-sol		 W4
21-Jan-2022 M1: Bonus Lecture: What is Life? M1 M1.1, M1.2
03-Feb-2022 written examination (13:30-16:30)
22-Apr-2022 resit written examination (18:00-21:00)

Study Guide

  • Please consult the Course Syllabus (lecture notes for 1st class) for advice on how to study the materials.

  • Each year there will be two written exam opportunities. You cannot bring notes or books to the written exam sessions. All needed formulas are supplied at the exam sheet.

Assignment

  • The assignment may be downloaded here. The Jupyter notebook explains the problem in detail.
  • Please hand in the completed notebook file on Canvas.
  • The solution to this year’s assignment can be found here.

Exam Preparation

In addition to the materials in the above table, we provide two representative practice written exams:

Instructors