High-Performance and AI Architectures

This course provides a structured, hands-on exploration of the principles and technologies behind high-performance computing (HPC) and artificial intelligence (AI), with a focus on efficient algorithm design and hardware acceleration for solving complex problems. Through a gradual buildup of foundational knowledge and applied skills, students will learn to optimize computational performance and implement advanced solutions for faster simulations and analyses. A key novelty of this course is the use of Python as a high-level language for rapid modeling and hardware acceleration in HPC and AI, enabling fast prototyping and optimization with modern libraries while maintaining technical depth.

The course begins with an introduction to parallel computing, where students explore key concepts such as scalability, memory overheads, algorithmic complexity, profiling, and benchmarking. Techniques like vectorization and just-in-time (JIT) compilation are taught alongside a hands-on lab focused on optimizing serial code. The next phase addresses CPU parallelism, covering multiprocessing and multithreading in depth. Practical labs teach students to implement and benchmark parallel CPU solutions. The course then transitions to GPU architectures, examining GPU internals, programming paradigms, and profiling tools. Students gain practical experience with GPU acceleration techniques in dedicated labs. The focus then shifts to AI architectures, where students explore both artificial and spiking neural networks, and their computational demands during training and inference. Practical implementation of AI workloads on advanced hardware is covered in accompanying labs on FPGA platforms and, if available, on dedicated AI accelerators such as Google's TPUs. The course concludes with integration and advanced topics, addressing scalability, memory management, and deployment considerations such as cost-efficiency in HPC/AI environments. Putting it all together, a final lab encourages students to take on the challenge of accelerating a real-world scientific problem, integrating all skills and tools acquired throughout the course.

• Lab assignments:

  • The lab component consists of six assignments, which together contribute 50% of the final grade.

  • Lab reports must be submitted on time, are graded independently, and the lab component must be passed as a whole in order to pass the course.

• Final exam:

  • The final exam is a randomized multiple-choice test conducted directly on Brightspace. It accounts for 50% of the final grade.

  • The exam is graded independently and is mandatory to pass the course.

• Final mark:

  • The final numerical grade (0–10) is calculated as the average of the lab assignment grade and the final exam grade.

    • Important: To pass the course, students must obtain at least 5.8/10 in both the lab assignments and the final exam individually.

  • As per standard TU Delft regulations, the overall final grade must also be ≥ 5.8/10 to pass the course.

• Exam resit:

  • A resit is only available to students who did not pass the final exam. In such cases, an oral resit exam is typically arranged in the weeks following the final exam and must be scheduled with the course instructor.

    • Note: There is no resit opportunity for lab assignments. If the lab component is failed, the course must be retaken.