Background and Course Description

This course will take the Bayesian statistical modeling approach to machine learning. Some of the key benefits of the Bayesian approach include the ability to quantify the uncertainty in the parameters/predictions through posterior probability distributions, the ability to incorporate prior knowledge in a principled way, the ability to learn the model hyperparameters and the right model size/complexity automatically from data, and the property of embodying online learning in a natural way. In this course, we will discuss the foundations of Bayesian modeling, especially in the context of machine learning and, through various case-studies/running-examples, we will look at how to set up a machine learning problem as a Bayesian model and how to design sampling/optimization techniques to perform computationally scalable inference in these models.

Pre-requisites

Instructor's consent. However, note that this course will make extensive use of concepts from probability, statistics, and optimization. Therefore a solid background on these topics, as well as introductory machine learning, will be essential. Students are also supposed to be familiar with programming in MATLAB/Python.

Grading

(Tentative break-up) There will be 4 homework assignments (total 25%) which may include a programming component, a mid-term (20%), a final-exam (25%), and a course project (25%). Class/Piazza participation will be worth 5% of your total grade.

Books

We will primarily use lecture notes/slides from this class. In addition, we will refer to monographs and research papers (from top Machine Learning conferences and journals) for some of the topics. Some recommended, although not required, books are: