Note: This site is currently under construction.The 2020/21 course "Bayesian Machine Learning and Information Processing" will start in November 2020 (Q2).
This course covers the fundamentals of a Bayesian (i.e., probabilistic) approach to machine learning and information processing systems. The Bayesian approach allows for a unified and consistent treatment of many model-based machine learning techniques. We focus on Linear Gaussian systems and will discuss many useful models and applications, including common regression and classification methods, Gaussian mixture models, hidden Markov models and Kalman filters. We will discuss important algorithms for parameter estimation in these models including the Expectation-Maximization (EM) algorithm and Variational Bayes (VB). The Bayesian method also provides tools for comparing the performance of different information processing systems by means of estimating the ``Bayesian evidence’’ for each model. We will discuss several methods for approximating Bayesian evidence. Next, we will discuss intelligent agents that learn purposeful behavior from interactions with their environment. These agents are used for applications such as self-driving cars or interactive design of virtual and augmented realities. Indeed, in this course we relate synthetic Bayesian intelligent agents to natural intelligent agents such as the brain. You will be challenged to code Bayesian machine learning algorithms yourself and apply them to practical information processing problems.
- Check the Canvas website regularly for new announcements.
- Prof.dr.ir. Bert de Vries (email: firstname.lastname@example.org) is the responsible instructor for this course and teaches all lectures with label B.
- Dr. Wouter Kouw (email@example.com) teaches all practical sessions on probabilistic programming with label W.
- Ismail Senoz, MSc (firstname.lastname@example.org), and Magnus Koudahl, MSc (email@example.com) are teaching assistants. Mr. Koudahl presents the “What is Life?" bonus lecture.
In principle, you can download all needed materials from the links below.
Please download the following books/resources:
- Christopher M. Bishop (2006), Pattern Recognition and Machine Learning. You can also buy a hardcopy, e.g. at bol.com.
- Ariel Caticha (2012), Entropic Inference and the Foundations of Physics.
- Bert de Vries et al. (2020), PDF bundle of lecture notes for lessons B0 through B12 (Ed. Q3-2019/20).
- The lecture notes may change a bit during the course, e.g., to process comments by students. A final PDF version will be posted after the last lecture.
The source files for the lecture notes are accessible at github. You can view all lecture notes online through the links below:
|11-Nov-2020||B0: Course Outline and Administrative Issues
B1: Machine Learning Overview
|13-Nov-2020||B2: Probability Theory Review||B2||B2.1, B2.2|
|18-Nov-2020||B3: Bayesian Machine Learning||B3||B3.1, B3.2|
|20-Nov-2020||W1: Probabilistic Programming 1 - Intro Bayesian ML|
|25-Nov-2020||B4: Factor Graphs and the Sum-Product Algorithm||B4||B4|
|27-Nov-2020||B5: Continuous Data and the Gaussian Distribution||B5||B5.1, B5.2, B5.3|
|02-Dec-2020||B6: Discrete Data and the Multinomial Distribution||B6||B6|
|04-Dec-2020||W2: ProbProg 2 - MP & Analytical Bayesian Solutions|
|11-Dec-2020||B8: Generative Classification
B9: Discriminative Classification
|B8, B9||B8, B9|
|16-Dec-2020||W3: ProbProg 3 - Regression and Classification|
|18-Dec-2020||B10: Latent Variable Models and Variational Bayes||B10||B10|
|06-Jan-2021||B11: Dynamic Models||B11||B11|
|08-Jan-2020||B12: Intelligent Agents and Active Inference||B12||B12|
|13-Jan-2020||W4: ProbProg 4: Latent Variable and Dynamic Models|
|15-Jan-2020||M1: Bonus Lecture: What is Life?||M1|
Each year there will be two exam opportunities. Check the official TUE course site for exam schedules. In the Q2-2020 course, your performance will be assessed by a WRITTEN EXAMINATION, which (very likely) will be offered both online (with proctoring software) and offline (on campus, if the situation allows it).
You cannot bring notes or books to the exam. All needed formulas are supplied at the exam sheet.
In preparation for the exam, we recommend that you work through the following exercises to test your understanding of the materials:
Please feel free to consult the following matrix and Gaussian cheat sheets (by Sam Roweis) when doing the exercises.
Prerequisites for this course: Mathematical maturity equivalent to undergraduate engineering program. Scientific programming skills (e.g. in Python, MATLAB or Julia) are helpful.
Here is a guide to help you install Julia and Jupyter, so that you can open and run the course notebooks on your own machine.