Research Seminar on Scalable Learning Systems

This seminar course aims to teach the students how to design and build parallel and distributed machine learning (ML) and deep learning (DL) solutions. The learning activities include paper reading, presentation, discussion, and project prototyping. We will provide a broad overview of the state-of-the-art parallel and distributed ML and DL algorithms and systems, with a strong focus on the scalability, resource efficiency, data requirements, and robustness of the solutions. We will cover ways of mapping state-of-the-art ML and DL solutions to massively-parallel AI accelerators such as GPUs. We will present an array of techniques for efficiently scaling ML and DL workloads to a large number of distributed nodes in the presence of system failures and malicious attacks. Lastly, we will cover methods for improving the scalability and the efficiency of deep generative learning approaches.

Course topics include

• Overview of parallel and distributed ML/DL algorithms

• Performance and scalability of state-of-the-art systems

• Hardware-accelerated ML/DL solutions

• Federated machine learning systems

• Deep generative learning systems

The final grade of the course consists of the following components:

• Group Presentation 1: project (weighting 20%)

• Group Report: project (weighting 40%)

• Homework assignments (weighting 20%)

• Group Presentation 1: paper (weighting 20%)

Final grade calculation = 0.2 * Group Presentation + 0.4 * Group Report + 0.2 * Homework Assignments + 0.2 * Group Presentation

A passing final grade for the course can only be earned when for all components at least a 5.0 is earned, and the weighted final grade is at least a 5.8.

In case of an insufficient final result, repair options may exist in accordance with Article 17A Times and number of examinations, sub 1, of the Teaching and Examination Regulations, for:

• Group Presentation project: Repair

• Group Report: Repair opportunity

• Homework assignments: Repair opportunity

• Group Presentation paper: Repair opportunity

Disclaimer: information may change depending on unforeseen circumstances or measures (see: TER Art 2, sub 5).