Course type:	Lecture + Exercise
Time:	Wednesday, 12:15 - 13:45, first meeting: Oct. 20
Location:	The course will be fully virtual/online. Weekly flipped classroom sessions will be held on Zoom. See ILIAS for Zoom link.
Organizers:	Frank Hutter, Abhinav Valada, Steven Adriaensen, Samuel Müller, Yash Mehta, Niclas Vödisch
Web page:	ILIAS (please also register for all elements of this course module in HISinOne)

Foundations of Deep Learning

Deep learning is one of the fastest growing and most exciting fields. This course will provide you with a clear understanding of the fundamentals of deep learning including the foundations to neural network architectures and learning techniques, and everything in between.

Course Overview

The course will be taught in English and will follow a flipped classroom approach. Every week there will be:
- a video lecture
- an exercise sheet
- a flipped classroom session (virtual/online, Wednesdays 12:15 - 13:45)
- an attendance optional exercise session (in-class/offline, Thursdays 10:15 - 11:45)
At the end, there will be a written exam (likely ILIAS test).
Exercises must be completed in groups and must be submitted a week (+ 1 day) after their release. Your submissions will be graded and you will receive weekly feedback. Your final grade will be solely based on a written examination, however a passing grade for the exercises is a prerequisite for passing the course.
Online course: All material will be made available online and course participation will not require in-person presence. That being said, we offer the opportunity for direct interaction with our tutors during weekly attendance optional in-class exercise sessions (building 82, HS 00-006). In addition, it is possible to attend the digital flipped classroom sessions on campus using your own laptop + headphones (Building 101 - HS 00 036).
Exam: The exam will likely be a test you complete on ILIAS. In person presence may be required (tba).

Course Schedule

The following are the dates for the flipped classroom sessions:
20.10.21 - Kickoff: Info Course Organisation / Team Formation
27.10.21 - Week 1: Overview of Deep Learning
03.11.21 - Week 2: From Logistic Regression to MLPs
10.11.21 - Week 3: Backpropagation
17.11.21 - Week 4: Optimization
24.11.21 - Week 5: Regularization
01.12.21 - Week 6: Convolutional Neural Networks (CNNs)
08.12.21 - Week 7: Recurrent Neural Networks (RNNs)
15.12.21 - Week 8: Practical Methodology & Architectures
22.12.21 - Week 9: Hyperparameter Optimization
12.01.22 - Week 10: Neural Architecture Search
19.01.22 - Week 11: Attention & Transformers
26.01.22 - Week 12: Auto-Encoders, Variational Auto-Encoders, GANs
02.02.22 - Week 13: Uncertainty in Deep Learning
09.02.22 - Round-up / Exam Q & A

The course material (lecture video, slides, exercise sheet) for "Week N" will be made available a week before the flipped classroom session for "Week N". For example, the material for Week 1 will be available on 20.10.21 and solutions to the exercises must be submitted latest 28.10.21 at 23:59. We will be using Zoom and the meeting link can be found on ILIAS in the "Flipped Classroom" folder.

In the first session (on 20.10.21) you will get additional information about the course and get the opportunity to ask general questions (and form groups!) While there is no need to prepare for this first session, we encourage you to already think about forming teams. The last flipped classroom session is held on 09.02.22.

Competition Results

This semester, we organised an optional student competition. In this challenge, students were to train a model to perform class prediction on a flower dataset (more info here).

There were two tracks:
- Fast-track (models with less than 100k parameters)
- Large-track (models with less than 25M parameters)
The winners per track were determined based on the accuracy of the submitted models on a hidden test set.

The Fast-track podium:
- 1st place: Nisarga Nilavadi Chandregowda, Pablo Marhoff, Tidiane Ndir (accuracy: 90.49%)
- 2nd place: Bijay Gurung, Caoting Li, Kartik Yadav (accuracy: 82.80%)
- 3rd place: Uygar Akkoc, Aron Bahram, Samir Garibov (accuracy: 73.12%)

The Large-track podium:
- 1st place: Paweł Bugyi, Abhijeet Nayak, Preethi Sivasankaran (accuracy: 95.15%)
- 2nd place: Akshay Chandra Lagandula, Sai Prasanna Raman, John Robertson (accuracy: 92.49%)
- 3rd place: Bijay Gurung, Caoting Li, Kartik Yadav (accuracy: 91.28%)

Congratulations!

Questions?

If you have a question, please post it in the ILIAS forum (so everyone can benefit from the answer).
Alternatively, you can also email dl-orga-ws21@cs.uni-freiburg.de