Course contents

The course covers the following topics:

  1. Fundamental of ML, e.g., scalability, batch size, etc.
  2. Distributed learning: data-parallelization
  3. Distributed learning: model-parallelization
  4. Robust learning, e.g., byzantine-resilient learning
  5. AutoML, e.g., hyperparameter optimization, meta learning, and neural architecture search
  6. ML platforms, e.g., BigDL, PyTorch Distributed, ZeRO



Intended learning outcomes

After passing the course, students should be able to:

  • Demonstrate systematic understanding of ML systems and capacity to scholarly analyze and criticize their components.
  • Reflect on the ideas and technologies related to ML systems with insight on their possibilities and limitations.
  • Examine how ML systems are currently used and evaluate how they can be used for new purposes and under different application domains.
  • Identify the need for further knowledge in improving ML systems.



Course Disposition

The course consists of six modules, where each module covers a different research area in ML systems. Each module has two sessions: one lecture session and one discussion session.

  • Lecture sessions: In the lecture session of each module, the teacher will introduce the context and give an overview of the reading material for the week. The students, then have a week to study the topic and go through the reading material. They are also required to submit a detailed review of the selected papers.
  • Discussion sessions: During the discussion session of each module, they review and discuss the topic and the papers in depth. The goal of this format is to both build a mastery of the material and also to develop a deeper understanding of how to evaluate and review research and hopefully provide insight into how to write better papers, identify open research questions and need for further research.



Examination and completion

The course will be assessed with a Pass/Fail grade, based on active participation in the discussion meetings, as well as a scientifically sound review report in each week. In addition to this, a passing student must attend at least 75% of all lectures and 75% of all student presentation sessions.



Recommended prerequisites

The target students of the course are mainly PhD students of the computer science, information and communication technology, and electrical engineering doctoral programmes, as well as all other PhD students who are interested to know the architecture and fundamentals of modern ML systems. The students should be familiar with the basics of ML, distributed systems, and have a good programming knowledge especially in Python or Scala.