Menu

Foundations of Deep Learning

Course type: Lecture + Exercise
Time: Wednesday, 14:15 - 15:45, first meeting: Nov. 4

Exception: Week 8, Thursday, 14:15 - 15:45, Jan. 7
Location: The course will be fully virtual/online.
Weekly flipped classroom sessions will be held on Zoom.

See ILIAS for Zoom link.
Organizers: Frank Hutter, Abhinav Valada, Joschka Bödecker, Steven Adriaensen, Tim Frederic Runge
Web page: ILIAS, HISinOne (lecture), HISinOne (exercises, please register for both!)

Foundations of Deep Learning

Deep learning is one of the fastest growing and most exciting fields. This course will provide you with a clear understanding of the fundamentals of deep learning including the foundations to neural network architectures and learning techniques, and everything in between.

Course Overview

The course will be taught in english and will follow a flipped classroom approach. Every week there will be:
- a video lecture
- an exercise sheet
- a flipped classroom session (virtual/online, Wednesdays 14:15-15:45) At the end, there will be a written exam (online ILIAS test). Exercises must be completed in groups and must be submitted a week (+ 1 day) after their release. Your submissions will be graded and you will receive weekly feedback. Your final grade will be solely based on a written examination, however a passing grade for the exercises is a prerequisite for passing the course. Online course: All material will be made available online and course participation will not require in-person presence.
Online exam: The exam is a test you complete on ILIAS and does not require in-person presence.

Course Schedule

The following are the dates for the release of video lectures: 04.11.20 - Week 1: Overview of Deep Learning
11.11.20 - Week 2: From Logistic Regression to MLPs
18.11.20 - Week 3: Backpropagation
25.11.20 - Week 4: Optimization
02.12.20 - Week 5: Regularization
09.12.20 - Week 6: Convolutional Neural Networks (CNNs)
16.12.20 - Week 7: Recurrent Neural Networks (RNNs)
06.01.21 - Week 8: Practical Methodology & Architectures (flipped classroom on Thursday 07.01!)
13.01.21 - Week 9: Hyperparameter Optimization
20.01.21 - Week 10: Neural Architecture Search
27.01.21 - Week 11: Auto-Encoders, Variational Auto-Encoders, GANs
03.02.21 - Week 12: Uncertainty in Deep Learning On the same day, there is a flipped classroom session about the material released the week before. An exception is Week 8, where the flipped classroom session on RNNs will take place on Thursday 07.01 (14:15-15:45) instead of Wednesday 06.01 (holiday). We will be using Zoom and the meeting link can be found on ILIAS in the Flipped Classroom folder. In the first session (on 04.11.20) you will instead get additional information about the course and get the opportunity to ask general questions (and form groups!) While there is no need to prepare for this first session, we encourage you to already think about forming teams. The first exercise on 'Week 1: Overview of Deep Learning' is due on 12.11.20 at 23:59. The last flipped classroom session is held on 10.02.21.

Competition Results

This semester, we organised an optional student competition. In this challenge, students were to train a model to perform class prediction on a flower dataset (more info here).
There were two tracks:
- Fast-track (models with less than 100k parameters)
- Large-track (models with less than 25M parameters)
The winners per track were determined based on the accuracy of the submitted models on a hidden test set.
The Fast-track podium:
- 1st place: Giulio Neusch-Frediani (accuracy: 76.25%)
- 2nd place: Guri Zabergja, Jeta Bekteshi, Albanot Makolli (accuracy: 72.75%)
- 3rd place: Florian Diederichs, Lukas Koenig, Nina Pant (accuracy: 71.75%)
The Large-track podium:
- 1st place: Guri Zabergja, Jeta Bekteshi, Albanot Makolli (accuracy: 98.5%)
- 2nd place: Florian Diederichs, Lukas Koenig, Nina Pant (accuracy: 97.0%)
- 3rd place: Samuel Boehm, Narges Dastanpour, Vytautas Jankauskas (accuracy: 95.0%)
Congratulations!

Questions?

If you have a question, please post it in the ILIAS forum (so everyone can benefit from the answer).
Alternatively, you can also email dl-orga-ws20@cs.uni-freiburg.de