Uni-Logo

Lecture: Automated Machine Learning

News

  • All news, slides and exercise sheets will be online at ILIAS
  • This course was formerly offered as "ML4AAD" (with a slight different focus). If you have successfully attended ML4AAD, please don't enrol for this lecture.

Background

Machine Learning (ML) and in particular Deep Learning (DL) achieved many breakthroughs in the last years. Unfortunately, ML/DL does not only require data, but it requires also a lot of expert knowledge, if you want to apply it successfully---even experts in ML/DL still need a lot of time to do it. Major challenges for new ML/DL applications include the choice of the algorithm to be used (SVM, random forest, deep neural networks) and its hyper-parameter settings (e.g., kernel coefficient of a RBF-SVM). Unfortunately, to obtain accurate predictions, these design decisions are crucial and have to be made for each dataset. This is particularly hard for deep learning, where we have to choose a well-performing architecture of the network and for example to set the hyper-parameters of the optimizer (e.g., learning rate). Since training deep neural networks often requires quite some time (minutes, hours or even weeks), we cannot exhaustively try several networks architectures and hyper-parameter configurations, but we have to find more efficient approaches. Overall, all these design decisions require a lot of expert knowledge, the process takes quite some time and the manual tuning is a tedious and error-prone task.

We will discuss approaches and meta-systems, that automate the process of obtaining well-performing machine learning systems, so-called Automated Machine Learning (AutoML). These AutoML systems allow for faster development of new ML/DL applications, require far less expert knowledge than doing everything from scratch and often even outperform human developers. In this lecture, you will learn how to use such AutoML systems, to develop your own systems and to understand ideas behind state-of-the-art AutoML approaches.

Requirements

We strongly recommend that you know the foundations of

  • machine learning (ML)
  • and deep learning (DL) in order to attend the course.

The participants should have attended at least one other course for ML and DL in the past.

Topics

The lectures are partitioned in several parts, including:

  • Hyper-parameter optimization
  • Neural Architecture Search
  • Meta-learning
  • Algorithm Control
  • Learning to learn & optimize

Dynamics

The course will be in English.

We will meet weekly for a lecture and an exercise. Roughly every week, there will be a new exercise sheet. Most exercises will be practical and involve teamwork (teams of 2 students!) so that you learn how to apply AutoML in practice. The exercises are no requirement for the exam (however highly recommended). However to get a grade in the end, you have to pass the exercises.

  • Lecture: Monday 14:15 (s.t.) - 15:45 in Building 106 SR 00 007
  • Exercise: Friday 14:15 (s.t.) - 15:45 in Building 106 SR 00 007

Exam

In the end, everyone (no teamwork!) has to implement a larger project which is the base of the final oral exam. In the first 15 minutes of the oral exam, you have to present your project and in the second 15 minutes, we will ask you questions about further course material.