Bayesian Nonparametrics
& Bayesian Machine Learning
Master MASH 2020-2021
Objectives
The course aims at providing a modern overview of Bayesian Nonparametric Statistics and Bayesian Machine Learning. Students will learn how to model statistical and machine learning problems with Bayesian nonparametric tools and study theoretical properties of the involved objects.
Syllabus
This course covers the fundamentals of Bayesian nonparametric inference and Bayesian machine learning and focuses on the key probabilistic concepts and stochastic modelling tools at the basis of the most recent advances in the field.
Part 1 Bayesian Nonparametrics
Discrete random probability measures Lecture notes (handwritten)
• Foundations of Bayesian nonparametric inference: exchangeability, de Finetti's representation theorem
• Clustering and Dirichlet process, random partitions
• Models beyond the Dirichlet process, random measures
• Latent features and the Indian buffet process
• Mixture models
Asymptotic aspects of Bayesian Nonparametrics Slides by Jean-Bernard Salomond
• Concentration
• Contraction rates
• Gaussian Process priors
Part 2 Bayesian Machine Learning
Validation
Students form groups. Each group reads and reports on a research paper from a list. I strongly encourage a dash of creativity: students should identify a weak point, shortcoming or limitation of the paper, and try to push in that direction. This can mean extending a proof, implementing another feature, investigating different experiments, etc.
Deliverables are a small report and a short oral presentation in front of the class, in the form of a student seminar, which will take place during the last lecture.
Student Seminar
Pad for paper choice
The students will study a paper by pair and present it during the last session (in March). Presentations will be about 20 minutes long. Papers can be chosen in one of the following ways:
- (mainly) in this list of seminal (or not) papers
- in the webpage Tutorials on Bayesian Nonparametrics, maintained by Peter Orbanz, which references some seminal Bayesian nonparametric papers
- by browsing https://scholar.google.fr/ or http://arxiv.org/ with keywords you are interested in
In the last two cases, please send me an email with the reference for approval.
Lecture notes
Lecture notes (handwritten) for Part 1, essentially based on the following bibliography.
Slides for Part 2.
Bibliography
- Jayanta Ghosh, and R. V. Ramamoorthi. 2003. Bayesian Nonparametrics. Springer Series in Statistics.
- Nils Hjort, Chris Holmes, Peter Müller, and Stephen Walker. 2010. Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.
- Peter Orbanz. 2012. Lecture Notes on Bayesian Nonparametrics. Available at: http://stat.columbia.edu/~porbanz/papers/porbanz_BNP_draft.pdf
- Bas Kleijn, Aad van der Vaart, Harry van Zanten. 2012. Lectures on Nonparametric Bayesian Statistics. Available at: https://staff.fnwi.uva.nl/b.j.k.kleijn/NPBayes-LecNotes-2015.pdf
Further links
Tutorials on Bayesian Nonparametrics, a webpage maintained by Peter Orbanz