Bayesian Inference and Computations

Lecturers

Professor Yanan Fan, CSIRO Data61 and UNSW
Dr Ke Sun, CSIRO Data61 and Australian National University

Synopsis

Introduction to Bayesian inference, the role of priors; pior/posterior computations; Point estimation, interval estimation and predictive distributions; Normal models, asymptotics; Monte Carlo methods/introduction to simulation; Markov chain Monte Carlo; reversible jump MCMC; Hamiltonian MCMC, approximate Bayesian inference , ML models, variational inference. Instructions with R and Python.

Course overview

Week 1: Introduction to Bayesian inference, the role of priors, pior/posterior computations, setting up R, use of R markdown, introduction to using R, point estimation, interval estimation and predictive distributions, normal models, asymptotics

Week 2: Monte Carlo methods/introduction to simulation, Markov chain Monte Carlo, Gibbs sampler/rates of convergence, convergence assessments

Week 3: Reversible jump MCMC, Hamiltonian Monte Carlo, introduction to approximate Bayesian computation (ABC)

Week 4: parametric space of ML models, information divergences, principal component analysis, autoencoders, reparametrization trick

Prerequisites

  • second year undergraduate level statistical inference
  • some familiarity with R/Python or similar computing software

This is intended as a foundation course.

Assessment

  • Four weekly assessments
    • Assessment 1 – 10%
    • Assessments 2-4 – 15% each
  • One final exam 45% (online, timed – 2hrs, open book)

(may be subject to change)

Attendance requirements

Participation in all lectures and tutorials is expected.

For those completing the subject for their own knowledge/interest, evidence of at least 80% attendance at lectures and tutorials is required to receive a certificate of attendance.

Resources/pre-reading

Some content for this course is drawn from a number of text books in order that you might use these for more detailed reading than is provided in the Lecture Notes. These sources are as follows:

  • Bayesian Data Analysis (second edition), A Gelman, J Carlin, H Stern and D Rubin, Chapman and Hall
  • Bayes and Empirical Bayes Methods for Data Analysis (second edition), B.P.Carlin and T.A.Louis, Chapman and Hall
  • Markov Chain Monte Carlo – Stochastic simulation for Bayesian inference, D. Gammerman, Chapman and Hall
  • Bayesian Inference, 2nd Edition, Vol 2B of “Kendall’s Advanced Theory of Statistics,” A. O’Hagan and J. J. Forster (2004), Arnold, London.
  • Machine Learning: A Probabilistic Perspective. Kevin P. Murphy. The MIT Press, 2011.

Not sure if you should sign up for this course?

Take this QUIZ to self-evaluate and get a measure of the key foundational knowledge required.

Professor Yanan Fan

Dr Yanan Fan is a Senior principal research scientist at CSIRO’s Data61 and an adjunct Professor at the School of Mathematics and Statistics, University of New South Wales where she has taught a range of courses in statistics for many years. Her primary research interest is in the development of Bayesian computational tools for the analysis of complex statistical models. In particular, she has worked on Markov Chain Monte Carlo, approximate Bayesian computational methods, as well as developing Bayesian semiparametric models, in particular Bayesian quantile regression methods.

Dr Ke Sun

Dr. Ke Sun is a principal research scientist in CSIRO’s Data61 and an honorary senior lecturer at ANU. His expertise is on the subjects of information geometry and the theory of deep learning. He has been publishing at premier machine learning conferences (ICML, NeurIPS, etc.). He serves in the international machine learning and information geometry community. He has been applying deep learning techniques and theories to various domains through his work at CSIRO.