High-Performance and AI Architectures

This course provides a structured, hands-on exploration of the principles and technologies behind high-performance computing (HPC) and artificial intelligence (AI), with a focus on efficient algorithm design and hardware acceleration for solving complex problems. Through a gradual buildup of foundational knowledge and applied skills, students will learn to optimize computational performance and implement advanced solutions for faster simulations and analyses. A key novelty of this course is the use of Python as a high-level language for rapid modeling and hardware acceleration in HPC and AI, enabling fast prototyping and optimization with modern libraries while maintaining technical depth.

The course begins with an introduction to parallel computing, where students explore key concepts such as scalability, memory overheads, algorithmic complexity, profiling, and benchmarking. Techniques like vectorization and just-in-time (JIT) compilation are taught alongside a hands-on lab focused on optimizing serial code. The next phase addresses CPU parallelism, covering multiprocessing and multithreading in depth. Practical labs teach students to implement and benchmark parallel CPU solutions. The course then transitions to GPU architectures, examining GPU internals, programming paradigms, and profiling tools. Students gain practical experience with GPU acceleration techniques in dedicated labs. The focus then shifts to AI architectures, where students explore both artificial and spiking neural networks, and their computational demands during training and inference. Practical implementation of AI workloads on advanced hardware is covered in accompanying labs on FPGA platforms and, if available, on dedicated AI accelerators such as Google's TPUs. The course concludes with integration and advanced topics, addressing scalability, memory management, and deployment considerations such as cost-efficiency in HPC/AI environments. Putting it all together, a final lab encourages students to take on the challenge of accelerating a real-world scientific problem, integrating all skills and tools acquired throughout the course.