Uni-Logo

Please note: We do not offer any summer internships. The theses and projects listed here are only available to students enrolled at the University of Freiburg. If you are not enrolled and are interested in working with our group, we encourage you to apply to our MSc programme. Information on this programme and how to apply can be found here: http://www.tf.uni-freiburg.de/studies/degree_programmes/master/msc_informatik.html

Getting in Touch about Projects and Theses

Since machine learning is currently one of the hottest topics around, our small group is flooded with requests, and we may not be able to offer projects to every interested student. Generally, it helps for finding a matching project if you have taken and excelled at our courses. All projects in our research group should be carried out in teams of at least 2 students (max. 4). Theses can also be done in synergy with theses/projects by others, but each student has to write their thesis by themselves, appropriately acknowledging work done by others. We strongly recommend doing a MSc project with our group before inquiring about a MSc thesis.

In order to make the selection process of students and topics as effective as possible, we ask potential candidates for projects/theses to send us an email (optimally, already as a team) to ml-student-inquiry@cs.uni-freiburg.de (or only to the person(s) in charge of a specific project you're interested in) with the following information (per team member):

1) Are you a BSc or MSc student? Which term? If BSc, are you planning on staying in Freiburg for a MSc?
2) Which ML-related courses have you taken?
3) Can you please attach your transcript of records?
4) Which projects have you done so far (in Freiburg and elsewhere)?
5) Which topics interest you most?

Since different projects require different skill sets, please also rate your skills in the following categories on a scale from ++ (very good) to -- (no knowledge/skill):
6. Creativity / ideas for developing new algorithms
7. Getting someone else's large code base to run
8. Running comprehensive experimental studies / keeping track of results
9. Self-motivation to push through even if things don't work for a while
10. Coding skills
         a. Python
         b. TensorFlow
         c. Keras
         d. PyTorch
         e. C/C++
11. Ability to read a RL paper, implement it and get it to work
12. Ability to read a DL paper, implement it and get it to work
13. Formal background, linear algebra
14. Formal background, proofs

For Hiwi job applications, please follow the same process as above.

Possible Topics

Here, we collect possible topics for BSc/MSc projects or theses. Please note that the list of projects below is typically incomplete, since we always have new ideas as time goes by and we do not always update the website right away. The most up-to-date list of projects can be found in the following (partially overlapping) presentations: Open student projects (Deep Learning course WS 2017/18) and Open student projects (Reinforcement learning course WS 2017/18. Below, you can find additional details for some projects.

Joint Architecture and Hyperparameter Search for Deep Neural Networks

Deep learning has celebrated many successes, establishing new state-of-the-art performance in many important domains (e.g., image recognition and speech recognition). However, applying it in practice can be challenging because it requires many crucial design decisions, such as choosing a neural network architecture and its hyperparameters. Recent advances in Automated Machine Learning (AutoML) have led to automated tools that can compete with machine learning experts on supervised learning tasks. However, current AutoML tools do not yet support modern neural networks effectively.

NAS and HPO for time-to-accuracy

So far, neural architecture search (NAS) and hyperparameter optimization (HPO) were mostly applied to achieve great performance in terms of accuracy (or similar generalization metrics). A new research direction is not only to achieve great accuracy, but also to achieve it fast---in some applications, you cannot afford to wait hours or days to train your neural network. To this end, a new competition, called DAWNbench, measured time-to-accuracy, i.e., how fast can a neural network achieve a pre-defined accuracy (e.g., 94% on CIFAR-10). In this project, we will study how we can build a NAS-HPO system that performs well wrt time-to-accuracy by applying techniques from algorithm configuration.

Requirements

  • Hands-on experience with deep learning (preferable on image data sets), e.g., PyTorch
  • Strong Python programming skills
  • Recommended: some background in algorithm configuration

Literature

Level: MSc

Contact: Marius Lindauer

Meta-Learning for AutoNet

One of our recent tools is AutoNet: automatically designed and configured deep neural networks. Although AutoNet can find very well-performing deep neural network, it requires sometimes quite some time to do so. In this project, we will study how we can learn from previous data sets how we should search for a well-performing deep neural network on a new data set. This approach has already proven to be effective in our well-known auto-sklearn tool.

Requirements

  • Hands-on experience with deep learning, e.g., PyTorch
  • Strong Python programming skills

Literature

Level: MSc (or very ambitious BSc students)

Contact: Marius Lindauer

AutoNet across Data Sets

One of our recent tools is AutoNet: automatically designed and configured deep neural networks. So far, AutoNet only searches for deep neural network on a single data set. In this project, we want to answer the question whether we can find well performing and robust architectures (and maybe also hyperparameters) across a set of data sets. If this is not possible, the question arises how much performance we would lose by a single architecture for a set of data sets.

Requirements

  • Hands-on experience with deep learning, e.g., PyTorch
  • Strong Python programming skills
  • Recommended: some background in algorithm configuration

Literature

Level: MSc or ambitious BSc students

Contact: Marius Lindauer

Algorithm Configuration and Hyperparameter Optimization

The performance of many algorithms depends on their parameter settings which have to be adjusted based on the set of input instances at hand. For example, the parameters of AI planners have to be changed based on the planning task at hand. Unfortunately, manual parameter tuning is a tedious and error-prone task. Therefore, automated algorithm configuration tools (e.g., our state-of-the-art tool SMAC) can help users by automatically optimizing the parameters of their algorithms.

Visualization of Configurator Footprints

To study the performance of search algorithms, users would like to see a visualization of how the algorithm traversed the search space. In this project, we focus on a special kind of algorithms, namely algorithm configurators. Algorithm configurators search in the parameter space of other algorithms to improve their performance. To visualize the search trajectory of such configurators, multidimensional scaling (MDS) was proposed, projecting the search space in 2D and then plotting it. However, MDS has some issues which we would like to address in the project (e.g., by learning an embedding via Auto-Encoders).

Requirements

  • Hands-on experience with deep learning
  • Strong Python programming skills
  • Recommended: some background in algorithm configuration

Literature

Level: MSc or BSc students

Contact: Marius Lindauer

Large-scale study on benchmarking continuous optimization problems

Progress in hyperparameter optimization relies on the availability of relevant and realistic benchmark problems. Unfortunately, these are often expensive to evaluate, e.g., in order to optimize the hyperparameters of a neural network one has to train and evaluate multiple networks. Having cheap-to-evaluate but meaningful problems can speed up rapid development and reveal limitations of different methods early. One category of such problems are artificial test functions, originally created for testing global optimization algorithms.

In this project, you will set up a suite to evaluate our algorithms on a broad set of such functions to identify their strengths and weaknesses.

Requirements

  • Basics in Bayesian optimization
  • Strong Python programming skills
  • Interest in running large empirical computational studies

Literature

Contact Katharina Eggensperger

Deep Reinforcement Learning for learning programs

Deep reinforcement learning is arguably the hottest topic in machine learning, and this set of projects will explore its use for learning and improving programs.

BLBT: Deep Learning for EEG Data

Decoding EEG data is a challenging task and has recently been approached with deep learning. All of the following projects aim to improve an existing decoding pipeline by applying recent state-of-the-art techniques.

Requirements

  • Understanding of neural networks
  • Strong Python programming skills
  • Understanding of EEG (optional, but beneficial)

Literature

Recurrent neural networks for decoding motor control data

Currently, our pipeline uses feedforward networks, which naturally do not capture information over time. As EEG data can be considered as a time series, a promising extension is to use recurrent neural networks instead.

Architecture Search

The current pipeline has been handcrafted and strongly builds on domain expert knowledge. This project aims at exploiting hyperparameter search to automatically design well performing decoders and with that supports and accelerate research.

Data Augmentation for EEG data

EEG data to train a model is usually scarce as one recording contains less than 1000 samples. One way to overcome this limitation is to do data augmentation by applying transformations to the training data and make the resulting model invariant to these transformations. Well performing transformations, such as rotation and rescaling are known to improve generalization performance for image classification, but for EEG data it is not clear which transformations will work. In this project you will create and evaluate possible transformations for EEG data.

Contact Robin Schirrmeister