Uni-Logo

Praktikum Bayesian Optimization

This Praktikum has the name Praktikum in der Arbeitsgruppe Automatisches Algorithmendesign: Automated Parameter Tuning in the HISinOne Campus Management system.

Background

Many algorithms in artificial intelligence and machine learning have free hyperparameters that can be modified to improve their performance in practice (e.g., numerical values like the learning rate in artificial neural networks or discrete algorithm flags like the training algorithm for a neural network). Setting these hyperparameters to the correct value often has a drastic impact. Throughout the last couple of years, researchers optimized these hyperparameters either manually, or used automated methods.

The first of the above-mentioned methods is tedious and unreliable, thus researchers developed several automatic tools to free themselves from manually adjusting hyperparameters. In this praktikum we will have a close look at Bayesian Optimization, a state-of-the-art hyperparameter optimization algorithm, implement it and apply it to several realistic problems.

Dynamics

The praktikum consists of two phases:

  1. In the first six weeks, you will implement the cornerstones of the Bayesian Optimization framework. To support this, we will give two lectures. One on Gaussian Processes, and a second on Bayesian Optimization
  2. During the rest of the semester, you will present and implement recent improvement to the Bayesian Optimization framework in teams of at least two people. More specific, we will briefly present the several possible topics in week seven. Your team will then present a ten-minute presentation of this topic, which will be given on the 15th of June. You then implement this technique and write a 2-page report about your experience and the outcomes. To make your knowledge available to all other participants of the praktikum, you will also give a ten-minute presentation in the last week of the lecture period.

We will hand out the assignments in the ILIAS system, this is also the place where you have to hand in your solutions. Please register early to our course in order to avoid technical problems.

Organization

  • Course type: Praktikum (6-ECTS)
  • Examination: Coding assignments (62.25%), presentations (25%) and final report (12.25%).
  • Requirements:
    • General programming skills. We will use python throughout this praktikum.
    • Knowledge of machine learning or statistical pattern recognition.
  • Language: English

Schedule

Date Topic

Mo, 20.04.15:

Introduction to course (mini-overview on Bayesian optimization);

Mo, 27.04.15:

Introduction to GPs
Please read chapter 2.1 and 2.2 of Gaussian Processes for Machine Learning as preparation (see link below)

Mo, 04.05.15:

Introduction to Bayesian Optimization (acquisition functions et al)

Mo, 11.05.15:

-

Mo, 18.05.15:

-

Mo, 25.05.15:

Pfingstmontag

Mo, 01.06.15:

Lecture/Introduction to advanced topics

Mo, 08.06.15:

-

Mo, 15.06.15:

Presentation of advanced topics

Mo, 22.06.15:

-

Mo, 29.06.15:

-

Mo, 06.07.15:

-

Mo, 13.07.15:

Hand in final assignment

Mo, 20.07.15:

Student presentations

Material

Bayesian Optimization tutorial

Assignments and slides can be found in the ILIAS system.

Slides from the kickoff meeting

Gaussian Processes for Machine Learning

For questions, please send an email to: fh@cs.uni-freiburg.de