Foundations of Deep Learning

Course type:	Lecture + Exercise
Time:	Monday, 14:15 - 15:45, first meeting: Oct. 17
Location:	The course will be Hybrid: - Weekly flipped classroom sessions will be held on Monday at HS 00 026 µ - SAAL (G.-Köhler-Allee 101) and via Zoom. See ILIAS for Zoom link. - Optional exercise sessions will take place on Friday 10:15-11:45 at HS 00 006 (G.-Köhler-Allee 082)
Organizers:	Frank Hutter, Abhinav Valada, Steven Adriaensen, Mahmoud Safari
Web page:	ILIAS (please make sure to also register for all elements of this course module in HISinOne)

Deep learning is one of the fastest growing and most exciting fields. This course will provide you with a clear understanding of the fundamentals of deep learning including the foundations to neural network architectures and learning techniques, and everything in between.

Course Overview

The course will be taught in English and will follow a flipped classroom approach.

Every week there will be:

a video lecture
an exercise sheet
a flipped classroom session (hybrid, Mondays 14:15 - 15:45)
an attendance optional exercise session (in-class/offline, Fridays 10:15 - 11:45)

At the end, there will be a written exam (likely an ILIAS test).

Exercises must be completed in groups and must be submitted a week (+ 1 day) after their release.
Your submissions will be graded and you will receive weekly feedback.
Your final grade will be solely based on a written examination, however, a passing grade for the exercises is a prerequisite for passing the course.

Hybrid course: All material will be made available online and course participation will not require in-person presence. That being said, we offer ample opportunity for direct interaction with the professors during live Q & A sessions (HS 00 026 µ - SAAL, G.-Köhler-Allee 101) and with our tutors during weekly attendance optional in-class exercise sessions (HS 00-006, G.-Köhler-Allee 082).

Exam: The exam will likely be a test you complete on ILIAS. In-person presence may be required (TBA).

Course Schedule

The following are the dates for the flipped classroom sessions:

17.10.22 - Kickoff: Info Course Organisation / Team Formation
24.10.22 - Week 1: Intro to Deep Learning
31.10.22 - Week 2: From Logistic Regression to MLPs
07.11.22 - Week 3: Backpropagation
14.11.22 - Week 4: Optimization
21.11.22 - Week 5: Regularization
28.11.22 - Week 6: Convolutional Neural Networks (CNNs)
05.12.22 - Week 7: Recurrent Neural Networks (RNNs)
12.12.22 - Week 8: Attention & Transformers
19.12.22 - Week 9: Practical Methodology
09.01.23 - Week 10: Hyperparameter Optimization
16.01.23 - Week 11: Neural Architecture Search
23.01.23 - Week 12: Auto-Encoders, Variational Auto-Encoders, GANs
30.01.23 - Week 13: Uncertainty in Deep Learning
06.02.23 - Round-up / Exam Q & A

The course material (lecture video, slides, exercise sheet) for "Week N" will be made available a week before the flipped classroom session for "Week N". For example, the material for Week 1 will be available on 17.10.22 and solutions to the exercises must be submitted latest 25.10.22 at 23:59. Virtual participation in flipped classroom sessions will be enabled using Zoom and the meeting link can be found on ILIAS in the "Flipped Classroom" folder.

In the first session (on 17.10.22) you will get additional information about the course and get the opportunity to ask general questions (and form groups!) While there is no need to prepare for this first session, we encourage you to already think about forming teams.
The last flipped classroom session will be held on 06.02.23.

Competition Results

Flower Classification Challenge

This semester, we organized an optional student competition. In this challenge, students were to train a model to perform class prediction on a flower dataset (more info here).

There were two tracks:
- Fast-track (models with less than 100k parameters)
- Large-track (models with less than 25M parameters)
The winners per track were determined based on the accuracy of the submitted models on a hidden test set.

The Fast-track podium:
- 1st place: Jelle Dehn, Soham Basu, Laura Neschen
- 2nd place: Adithya Anoop Thoniparambil, Daniel Rogalla
- 3rd place: Dominika Matus, M'Saydez Campbell, Florian Vogt

The Large-track podium:
- 1st place: Muhammad Ali
- 2nd place: Jelle Dehn, Soham Basu, Laura Neschen
- 3rd place: Premnath Srinivasan, Rishabh Verma, Ali Sarlak

Congratulations!

Tree Segmentation Challenge

After the success of the flower classification challenge, we organized a second optional student competition, spanning two semesters. In this challenge, students were to train a model to perform semantic segmentation on a tree dataset (more info here).

The podium:
- 1st place: Rishabh Verma, Premnath Srinivasan
- 2nd place: Ali Sarlak, Elham Elyasi
- 3rd place: Noah Lenagan

Congratulations!

Questions?

If you have a question, please post it in the ILIAS forum (so everyone can benefit from the answer).

Alternatively, you can also email dl-orga-ws22@cs.uni-freiburg.de