Bayesian Statistics

Objectives

The course aims at providing an overview of Bayesian statistics. Students will learn how to model statistical and machine learning problems from a Bayesian perspective and study some theoretical properties of the models.

Where and when

At Ensimag, 681 rue de la Passerelle, St Martin d'Heres.

September to November.

Syllabus

This course is in two parts first covering fundamental aspects of Bayesian inference, and then moving to more advanced topics.

Part 1 (following Peter Hoff's textbook: A first course in Bayesian statistical methods)

• Foundations of Bayesian inference: exchangeability, de Finetti's representation theorem

• Conjugacy in simple models (binomial, Poisson, Gaussian)

• Some elements of posterior sampling, Markov chain Monte Carlo

Part 2 (advanced topics)

• Bayesian nonparametrics (handwritten Lecture notes)

  • Clustering and Dirichlet process, random partitions

  • Models beyond the Dirichlet process, random measures, Indian buffet process

  • Gaussian processes

  • Some elements of Bayesian asymptotics

• Bayesian deep learning

  • Bayesian neural networks and their Gaussian process limit

• Bayesian bandits (chapters 34 & 35 of Lattimore, T., and Szepesvari, C. (2022). Bandit Algorithms)

Course progress

Here I'll keep note of which chapters/topics have been dealt with in the past courses:

1. Hoff's Chapter 1

2. Hoff's Chapters 2 & 4 (introducing some Monte Carlo)

3. Hoff's Chapter 3 (Bayes toolbox: predictive distribution, credible regions)

4. 
   

Bibliography

• Hoff, P. D. (2009). A first course in Bayesian statistical methods. Springer Science & Business Media. Book webpage

• Neal, R. M. (2012). Bayesian learning for neural networks (Vol. 118). Springer Science & Business Media.

• Hjort, N. L., Holmes, C., Müller, P., & Walker, S. G. (2010). Bayesian nonparametrics. Cambridge series in statistical and probabilistic mathematics. Cambridge: Cambridge Univ. Press.

• Orbanz, P. (2012). Lecture Notes on Bayesian Nonparametrics. Available at: http://stat.columbia.edu/~porbanz/papers/porbanz_BNP_draft.pdf

• Kleijn, B., van der Vaart, A., & van Zanten, H. (2012). Lectures on Nonparametric Bayesian Statistics. Available at: https://staff.fnwi.uva.nl/b.j.k.kleijn/NPBayes-LecNotes-2015.pdf

• Lattimore, T., and Szepesvari, C. (2022). Bandit Algorithms. Link.

Further links

• Bayesian Deep Learning: NeurIPS workshops

• Tutorials and Videolectures

• Books and Chapters

• Tutorials on Bayesian Nonparametrics, a webpage maintained by Peter Orbanz

• BNPdensity, an R package for modeling mixtures with normalized random measures.