Uni-Logo

Advanced Deep Learning

The seminar language will be English (even if everyone is German-spoken, to practice presentation skills in English). The first meeting takes place on April 24th.

For projects, have a look at our open projects site.

Topics

We will go through the 3rd part of the deep learning book and selected recent research papers which were published since the release of the book. We will release an exact list of papers and book chapters during our first meeting on April 24th.

Recommended Background Knowledge

  • Deep learning lab course
  • Machine learning or statistical pattern recognition
  • Reinforcement learning

Procedure

We will meet weekly (Tuesday 14:15-15:45, in building 106, room SR 00-007) to discuss research papers from the list of available papers below. Every week, one student presents a paper and leads the following discussion. All other participants read the paper and submit a summary, answer a given question and ask 1-3 questions due the Monday before. The presenter will have access to the questions and is expected to take them into account during the presentation. All participants will discuss the paper, its merits, and limitations. This discussion will, in part, be guided by the questions submitted by the participants. At the end of the week the presenter hands in a two page report on the paper. The final grade takes the oral presentation and the written report into account.

The final grade takes the oral presentation, the written report, the quality of summaries and questions submitted, and class participation into account.

Besides the seminar topic, you will learn several skills necessary not only in academia:

  • read and understand research papers
  • assessing the strengths and weaknesses
  • oral presentation in front of your peers
  • discussion with your peers
  • high level summary of research with which you are not intimately familiar.

Schedule

Date: Topic 1 Topic 2
15.05.2018 Auto-Encoders Variational Auto-Encoders (Paper)(Tutorial)(Deep Learning Book; Section 20.10.3)
29.05.2018 Bayesian methods for Neural Networks Bayes by Backprop
05.06.2018 Generative Adversarial Networks Generative Adversarial Networks 2
Further reading: (1) General tutorial on Generative Adversarial Networks by Ian Goodfellow: Youtube, PDF (2) Section 20.10.4 of the Deep Learning (Book)
12.06.2018 Deep Generative Networks 1(20.1-20.4) Deep Generative Networks 2(20.5-20.8 and 20.14)

What to put into the final report?

In a nutshell, we think of this report as a detailed summary of the paper you presented that also covers points that would come up in a research discussion about the paper. (We say "paper", here even though, you are not restricted to only write about the paper you present.)

E.g., some questions that should be discussed at some point in the report next to a detailed summary are the following:

  • What is the paper's main contribution and why is it important?
  • How does it relate to other techniques in the literature?
  • What are strong and what are weak points about the paper?
  • What would be interesting follow-up work? Any possible improvements in the methods? Any further interesting applications?
  • Is the code/data available online? Does it run off-the-shelf? If not, what problems are there with running it? (You should only put a limited amount of time into this; not longer than a full work day.)

Formatting and length of the final report

Final reports have to be typeset in LaTeX (sorry, but you were warned). We will use the formatting guidelines and electronic templates from the ICML machine learning conference. Reports don't have to be long (you already wrote all the paper summaries); 2 pages in ICML style are appropriate. Do not go beyond 4 pages - you might not be able to include everything you would like to include, but that is common in academic writing.

Material

Contact

For questions, please email us: kleinaa@cs.uni-freiburg.de, feurerm@cs.uni-freiburg.de, fh@cs.uni-freiburg.de