Research Seminar on Scalable Learning Systems

This seminar course aims to teach the students how to design and build parallel and distributed machine learning (ML) and deep learning (DL) solutions. The learning activities include paper reading, presentation, discussion, and project prototyping. We will provide a broad overview of the state-of-the-art parallel and distributed ML and DL algorithms and systems, with a strong focus on the scalability, resource efficiency, data requirements, and robustness of the solutions. We will cover ways of mapping state-of-the-art ML and DL solutions to massively-parallel AI accelerators such as GPUs. We will present an array of techniques for efficiently scaling ML and DL workloads to a large number of distributed nodes in the presence of system failures and malicious attacks. Lastly, we will cover methods for improving the scalability and the efficiency of deep generative learning approaches.

Course topics include

• Overview of parallel and distributed ML/DL algorithms

• Performance and scalability of state-of-the-art systems

• Hardware-accelerated ML/DL solutions

• Federated machine learning systems

• Deep generative learning systems