Fundamental Research in Machine and Deep Learning

This course is about doing fundamental empirical research in machine/deep learning. It is about doing research to better understand ML/DL models, what assumptions they make, when they fail, how to pose research questions, answer empirical hypotheses experimentally and analyze results.

Machine/deep learning research goes hand-in-hand with designing applications, superior performance, more data, improved accuracy, faster inference, bigger models, better algorithms, solving a problem, creating a new artifact/tool/AI, etc. While these concepts are all valuable in their own right, they are NOT the topic of this course. This course is NOT about applying ML/DL to "solve" some problem. Instead, here, we study how to "understand" and do fundamental empirical research in ML/DL itself. This course is about repeatedly asking "Why?"; it's about better understanding, reflecting, questioning, reproducing, and analyzing ML/DL research.

Students will

• read, present and debate scientific papers on machine learning scholarship,

• construct a logical research "storyline", see: https://jvgemert.github.io/storyline.pdf

• create synthetic controlled experiments, see: https://controlledexperimentsinml.org/

• reproduce existing ML/DL research, see https://reproducedpapers.org/

  while honing a critical attitude and being able to communicate concisely and clearly.

The final grade of the course consists of the following components:

• Group Presentation: presentation about an assigned paper on research methodology (weighting 10%)

• Individual Report 1: storyline (weighting 15%)

• Individual Report 2: toy problem (weighting 15%)

• Group Report: reproduction blog post about reproducing (a part) of a ML/DL paper, each student has an individual part in the reproduction (weighting 60%)

Final grade calculation = 0.1 * Group Presentation + 0.15 * Individual Report 1 + 0.15 * Individual Report 2 + 0.6 * Group Report

A passing final grade for the course can only be earned when for all components at least a 5.0 is earned, and the weighted final grade is at least a 5.8.

In case of an insufficient final result, repair options may exist in accordance with Article 17A, Times and number of examinations, sub 1, of the Teaching and Examination Regulations, for:

• Group Presentation: Resit opportunity

• Individual Report 1: Repair opportunity

• Individual Report 2: Repair opportunity

• Group Report: Repair opportunity

Disclaimer: information may change depending on unforeseen circumstances or measures (see: TER Art 2, sub 5).