Bayesian nonparametric course - Master MASH 2019

Objectives

The course aims at providing a modern overview of Bayesian nonparametric statistics. Students will learn how to model statistical and machine learning problems with Bayesian nonparametric tools and study theoretical properties of the involved objects.

Syllabus

This course covers the fundamentals of Bayesian nonparametric inference and focuses on the key probabilistic concepts and stochastic modelling tools at the basis of the most recent advances in the field.

Part 1 Discrete random probability measures (Julyan Arbel) Lecture notes (handwritten)

• Foundations of Bayesian nonparametric inference: exchangeability, de Finetti's representation theorem

• Clustering and Dirichlet process, random partitions

• Models beyond the Dirichlet process, random measures

• Latent features and the Indian buffet process

• Mixture models

Part 2 Asymptotic aspects of Bayesian nonparametrics (Jean-Bernard Salomond) slides

• Concentration

• Contraction rates

• Gaussian Process priors

Grade

The final grade on 20 will consist of

• 10 points for Exercises

• 10 points for for the study of a Research article (Mini Seminar)

Exercises

You are asked to solve 8 exercises from Bas Kleijn, Aad van der Vaart, Harry van Zanten. 2012. Lectures on Nonparametric Bayesian Statistics. From as many different chapters in total as possible. For March 15.

Send the exercises by email: julyan.arbel+bnp@gmail.com and jean-bernard.salomond@u-pec.fr. You can type them on LaTeX, possibly using knitr or RMarkdown especially if you intend to code in R. Scanned handwritten notes are fine as well.

Mini Seminar

Pad for paper choice

The students will study a paper by pair and present it during the last session (in March). Presentations will be about 20 minutes long. Papers can be chosen in one of the following ways:

- (mainly) in this list of seminal (or not) papers

- in the webpage Tutorials on Bayesian Nonparametrics, maintained by Peter Orbanz, which references some seminal Bayesian nonparametric papers

- by browsing https://scholar.google.fr/ or http://arxiv.org/ with keywords you are interested in

In the last two cases, please send us an email with the reference for approval.

The presentation should stress the main contribution(s) of the article, see the evaluation criteria.

There will possibly be 2 bonus points for an optional written report of the article (max two pages): this report should aim at mimicking a referee report. See here for some useful hints: JRSS referee tips, NIPS Reviewer best practices, Institute of Mathematical Statistics - Guidelines for Referees.

Lecture notes

Lecture notes (handwritten) for Part 1, essentially based on the following bibliography.

Slides for Part 2.

Bibliography

- Jayanta Ghosh, and R. V. Ramamoorthi. 2003. Bayesian Nonparametrics. Springer Series in Statistics.

- Nils Hjort, Chris Holmes, Peter Müller, and Stephen Walker. 2010. Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.

- Peter Orbanz. 2012. Lecture Notes on Bayesian Nonparametrics. Available at: http://stat.columbia.edu/~porbanz/papers/porbanz_BNP_draft.pdf

- Bas Kleijn, Aad van der Vaart, Harry van Zanten. 2012. Lectures on Nonparametric Bayesian Statistics. Available at: https://staff.fnwi.uva.nl/b.j.k.kleijn/NPBayes-LecNotes-2015.pdf

Further links

Tutorials and Videolectures

Books and Chapters

Tutorials on Bayesian Nonparametrics, a webpage maintained by Peter Orbanz

DPpackage: Bayesian Semi- and Nonparametric Modeling in R, an R package by Alejandro Jara

BNPdensity, an R package for modeling mixtures with normalized random measures, by Ernesto Barrios, Antonio Lijoi, Luis E. Nieto-Barajas and Igor Prünster