“I enjoyed the social aspect the most. There were so many intelligent and motivated young mathematicians—and now I know them! Although it was only one month, I do believe I have formed life-long friendships.”

Cassady Swinburne, University of Tasmania

The Mathematical Engineering of Deep Learning

Lecturers

Associate Professor Yoni Nazarathy, The University of Queensland
Professor Benoit Liquet, Macquarie University and UPPA, France
Dr. Sarat Moka, The University of Queensland

Synopsis

In the last few years deep learning has seen explosive growth and even dubbed as the “new electricity”. This is due to its incredible success in transforming and improving a variety of automated applications. At its core, deep learning is a collection of models, algorithms, and techniques, such that when assembled together, efficient automated machine learning is executed. The result is a method to create trained models that are able to detect, classify, translate, create and take part in systems that execute human like tasks and beyond.

In this course we focus on the mathematical engineering aspects of deep learning. For this we survey and investigate the collection of algorithms, models, and methods that allow the statistician, mathematician, or machine learning professional to use deep learning methods effectively. Many machine learning courses focus either on the practical aspects of programming deep learning, or alternatively on the full development of machine learning theory, only presenting deep learning as a special case. In contrast, in this course, we will aim to focus directly on deep learning methods, understanding the engineering mathematics that drives this field.

Course Overview

  • A student completing this course will possess a solid understanding of the fundamental models, algorithms, and techniques of deep learning. These include feedforward networks, convolutional networks, recurrent neural networks, autoencoders, generative adversarial networks, first order methods of learning (optimization), second order method of learning, regularization techniques, and general benchmarking methods.Students will also gain some hands on experience with machine learning frameworks such as TensorFlow (R and Python), Keras (Python), PyTorch (Python), and Flux (Julia).

Prerequisites

  • Working knowledge of multi-variate calculus, linear algebra, probability, and statistics is assumed. Basic ability to program in at least one scientific programming language such as Python, R, Matlab, or Julia is needed as well. Review resources will be supplied prior to the course.

Assessment

  • There are 3 quizzes worth 12% each. Each quiz is a 45 minute in-class quiz. The quizzes allow for a single page summary of the material.
  • There are two assignments that are to be worked on during the period of the summer school. They are each worth 16%. These assignments involve computational aspects of deep learning. They are to be carried out in pairs and in certain authorized cases in groups of three.
  • Finally, there is a bigger individual (not in pairs) final assignment (project) worth 32%. A checkpoint of that project including a summary and plan is due in the last week of the summer school. The check point is worth 6% out of the total grade (or 6/32 of the project). It involves a two-page proposal. The remainder of the project is due during the exam period of the summer school and is to be worked on individually after the summer school finishes. The project is worth 26% out of the total grade (or 26/32 of the project).

(Assessment subject to change)

Resources

Course materials are available here.

Lecture notes will be provided during the course.

Pre-Course Quiz

Not sure if you should sign up for this course?

Take the-course quiz here.

Solution available here.

yoni-nazarathy

Associate Professor Yoni Nazarathy
University of Queensland

Associate Professor Yoni Nazarathy from the School of Mathematics and Physics, specialises in data science, probability and statistics. His specific research interests include scheduling, control, queueing theory, and machine learning. He has been at UQ for nearly a decade, teaching courses at UQ’s Masters of Data Science program and working on research. Prior to his previous academic positions in Melbourne and the Netherlands, he worked in the aerospace industry in Israel. In recent years, he has also been heavily involved with primary and secondary mathematics education and is the co-founder of an EdTech mathematics organisations called One on Epsilon. Also, he is the co-author of an introductory data science book: Statistics with Julia. Like many other mathematics and physics academics, he took great interest in epidemics at the start of COVID-19 and in addition to his co-authored Safe Blues program, he co-organises a weekly pandemic (Zoom) seminar at the School of Mathematics and Physics.

benoit-liquet

Professor Benoit Liquet
Macquarie University and UPPA, France

Benoit Liquet is Professor of Mathematical and Computational Statistics at Macquarie University in the Department of Mathematics and Statistics. In addition he is affiliated to the Université de Pau et Pays de l’Adour (UPPA) and was previously affiliated with ACEMS (Centre of Excellence for Mathematical and Statistical Frontiers), Queensland University of Technology. He was a senior lecturer in Statistics at The University of Queensland (from 2013-2015), Senior Investigator Statistician at Medical Research Council Biostatistics Unit in Cambridge (from 2012-2013), Associate Professor at Bordeaux University (from 2017-2012). Throughout his career he has extensively worked in developing novel statistical models mainly to provide novel tools to analyse clinical, health and biological data arising from epidemiological studies. More recently (since 2011), he moved to the field of computational biology and generalised some of these methods so that they scale to high throughput (“omic”) data. He has been teaching an advanced course on machine learning and high dimensional data at the UPPA. Benoit Liquet works on Applied Statistics, as well as on the development of R packages and on industrial applications (such as Machine Learning).

sarat-moka-cropped

Dr. Sarat Moka
University of Queensland

Dr Sarat Babu Moka is an ACEMS Postdoctoral Research Fellow in the School of Mathematics and Physics at The University of Queensland since 2017. He obtained his PhD in System Science from Tata Institute of Fundamental Research, Mumbai, India. Prior to his PhD, he worked as a scientist at the Indian Space Research Organization. His research interests broadly lie in Applied Probability with a specific focus on Monte Carlo Simulation and Machine Learning. Over the past one year, he has been teaching an advanced course on “Problems & Applications in Modern Statistics” (STAT3500) at The University of Queensland. He has co-authored Safe Blues program for estimation and control in the fight against COVID-19.