The field of tabular das has recently been exploding with advances through large language models (LLMs), deep learning algorithms, and foundation models. In this seminar, we want to dive deep into these very recent advances to understand them.
Course type: | Seminar |
Time | Five slots, to be determined with all participants. Kick-off is likely on the 23rd of October at 10 to 11 am. |
Location | in-person; Meeting Room in our ML Lab |
Organizers | Lennart Purucker |
Registration | Via HISinOne (maximal six students, registration opens 14th of October) |
Language | English |
Prerequisites
We require that you have taken lectures on or are familiar with the following:
- Machine Learning
- Deep Learning
- Automated Machine Learning
Organization
After the kick-off meeting, everyone is assigned a paper about recent advances in deep learning (one or multiple papers, depending on the content). Then, everyone is expected to understand and digest their assigned papers and prepare two presentations. The first presentation is given in midterms (two separate slots), and the second during the endterms (two separate slots).
- The first presentation will focus on the relationship between the papers, any relevant related work, any background to understand the paper, and the greater context of the work.
- The second presentation will focus on the paper's contributions, describing them in detail.
In addition to the second presentation, students are expected to contribute an "add-on" related to the paper for the final report. This includes but is not limited to reproducing some experiments, implementing a part of the paper, providing a greater literature survey, fact-checking citations, experiments, or methodology, building a WebUI or demo for the paper, etc. Students can (e-)meet with Lennart Purucker for feedback and any questions (e.g., to discuss a potential "add-on").
Grading
- Presentations: 40% (two times 20min + 20min Q&A)
- Report: 40% (4 pages in AutoML Conf format, due one week after last end term)
- Add-on: 20%
Short(er) List of Potential Papers / Directions:
LLMs
- https://arxiv.org/abs/2409.03946
- https://arxiv.org/abs/2403.20208
- https://arxiv.org/abs/2404.00401, https://aclanthology.org/2024.lrec-main.1179/, https://arxiv.org/abs/2408.09174
- https://arxiv.org/abs/2404.05047
- https://arxiv.org/abs/2404.17136
- https://arxiv.org/abs/2404.18681, https://arxiv.org/abs/2405.17712, https://arxiv.org/abs/2406.08527
- https://arxiv.org/abs/2405.01585
- https://arxiv.org/abs/2407.02694
- https://arxiv.org/abs/2408.08841
- https://arxiv.org/abs/2408.11063
- https://arxiv.org/abs/2403.19318
- https://arxiv.org/abs/2403.06644
- https://arxiv.org/abs/2402.17453, https://arxiv.org/abs/2409.07703
- https://arxiv.org/abs/2403.01841
Deep Learning
- https://arxiv.org/abs/2405.08403
- https://arxiv.org/abs/2307.14338
- https://arxiv.org/abs/2305.06090, https://arxiv.org/abs/2406.00281
- https://arxiv.org/pdf/2404.17489
- https://arxiv.org/abs/2405.14018, https://arxiv.org/abs/2406.05216, https://arxiv.org/abs/2406.17673, https://arxiv.org/abs/2409.05215, https://arxiv.org/abs/2406.14841
- https://arxiv.org/abs/2408.06291
- https://arxiv.org/abs/2408.07661
- https://arxiv.org/abs/2409.08806
- https://arxiv.org/abs/2404.00776
Foundation Models / In-Context Learning
- https://arxiv.org/abs/2406.09837, https://arxiv.org/abs/2307.09249
- https://arxiv.org/abs/2406.06891,https://arxiv.org/abs/2408.17162
- https://arxiv.org/abs/2405.16156, https://arxiv.org/abs/2402.11137 , https://arxiv.org/pdf/2406.05207
- https://arxiv.org/abs/2403.05681,https://arxiv.org/abs/2408.02927
- https://arxiv.org/abs/2407.21523
Multi-Modal