Prerequisites
We require that you have taken lectures on or are familiar with the following:
- Machine Learning
- Deep Learning
- Automated Machine Learning
Organization
After the kick-off meeting, everyone is assigned a paper about recent advances in deep learning (one or multiple papers, depending on the content). Then, everyone is expected to understand and digest their assigned papers and prepare two presentations. The first presentation is given in midterms (two separate slots), and the second during the endterms (two separate slots).
- The first presentation will focus on the relationship between the papers, any relevant related work, any background to understand the paper, and the greater context of the work.
- The second presentation will focus on the paper's contributions, describing them in detail.
In addition to the second presentation, students are expected to contribute an "add-on" related to the paper for the final report. This includes but is not limited to reproducing some experiments, implementing a part of the paper, providing a greater literature survey, fact-checking citations, experiments, or methodology, building a WebUI or demo for the paper, etc. Students can (e-)meet with Lennart Purucker for feedback and any questions (e.g., to discuss a potential "add-on").
Grading
- Presentations: 40% (two times 20min + 20min Q&A)
- Report: 40% (4 pages in AutoML Conf format, due one week after last end term)
- Add-on: 20% (due with the report)
Short(er) List of Potential Papers / Directions:
LLMs
- https://arxiv.org/abs/2409.03946
- https://arxiv.org/abs/2403.20208
- https://arxiv.org/abs/2404.00401, https://aclanthology.org/2024.lrec-main.1179/, https://arxiv.org/abs/2408.09174
- https://arxiv.org/abs/2404.05047
- https://arxiv.org/abs/2404.17136
- https://arxiv.org/abs/2404.18681, https://arxiv.org/abs/2405.17712, https://arxiv.org/abs/2406.08527
- https://arxiv.org/abs/2405.01585
- https://arxiv.org/abs/2407.02694
- https://arxiv.org/abs/2408.08841
- https://arxiv.org/abs/2408.11063
- https://arxiv.org/abs/2403.19318
- https://arxiv.org/abs/2403.06644
- https://arxiv.org/abs/2402.17453, https://arxiv.org/abs/2409.07703
- https://arxiv.org/abs/2403.01841
Deep Learning
- https://arxiv.org/abs/2405.08403
- https://arxiv.org/abs/2307.14338
- https://arxiv.org/abs/2305.06090, https://arxiv.org/abs/2406.00281
- https://arxiv.org/pdf/2404.17489
- https://arxiv.org/abs/2405.14018, https://arxiv.org/abs/2406.05216, https://arxiv.org/abs/2406.17673, https://arxiv.org/abs/2409.05215, https://arxiv.org/abs/2406.14841
- https://arxiv.org/abs/2408.06291
- https://arxiv.org/abs/2408.07661
- https://arxiv.org/abs/2409.08806
- https://arxiv.org/abs/2404.00776
Foundation Models / In-Context Learning
- https://arxiv.org/abs/2406.09837, https://arxiv.org/abs/2307.09249
- https://arxiv.org/abs/2406.06891,https://arxiv.org/abs/2408.17162
- https://arxiv.org/abs/2405.16156, https://arxiv.org/abs/2402.11137 , https://arxiv.org/pdf/2406.05207
- https://arxiv.org/abs/2403.05681,https://arxiv.org/abs/2408.02927
- https://arxiv.org/abs/2407.21523
Multi-Modal