Speaker Information and Talk Abstracts

Tuesday, 10th June 2025

Bayesian Optimization and Hyperparameter Optimization

<TBA>

Bio. Bernd Bischl holds the chair of Statistical Learning and Data Science at the Department of Statistics at the Ludwig-Maximilians-University Munich and is a co-director of the Munich Center for Machine Learning (MCML), one of Germany’s national competence centers for ML. He studied Computer Science, Artificial Intelligence and Data Sciences in Hamburg, Edinburgh and Dortmund and obtained his Ph.D from Dortmund Technical University in 2013 with a thesis on “Model and Algorithm Selection in Statistical Learning and Optimization”. His research interests include AutoML, model selection, interpretable ML, as well as the development of statistical software.

He is a member of ELLIS in general, and a faculty member of ELLIS Munich, an active developer of several R-packages, leads the mlr (Machine Learning in R) engineering group and is co-founder of the science platform OpenML for open and reproducible ML.

Furthermore, he leads the Munich branch of the ADA Lovelace Center for Analytics, Data & Application, i.e. a new type of research infrastructure to support businesses in Bavaria, especially in the SME sector.

Bernd Bischl

Professor for Statistical Learning and Data Science at LMU Munich & Munich Center of Machine Learning

Marius Lindauer

Professor of Machine Learning at Leibniz University Hannover & L3S Research Center

Intro to AutoML

This talk offers a broad introduction to Automated Machine Learning (AutoML), an area that's rapidly changing how we approach AI development. We'll explore how AutoML aims to automate various stages of the machine learning workflow, from initial data preparation to the final deployment of models. The session will highlight the overarching benefits of adopting AutoML, such as making advanced AI techniques more accessible and speeding up the process of building effective models. We'll touch upon some of the core ideas that power AutoML systems and examine how they're being applied.

Bio. Prof. Dr. Marius Lindauer has been a professor of Machine Learning at Leibniz University Hannover since 2019. He completed his PhD in 2015 at the University of Potsdam under the supervision of Prof. Dr. Thorsten Schaub and Prof. Dr. Holger Hoos. Prior to his current position, he held a PostDoc role and later was a junior research group lead at the University of Freiburg, working with Prof. Dr. Frank Hutter. He co-leads AutoML.org and AutoML-space and co-founded the research network COSEAL, the AutoML Conference, and the Institute of AI at Leibniz University Hannover. His work has garnered recognition through several international competition wins, notably in Automated Machine Learning. In 2022, Prof. Lindauer was awarded the prestigious ERC Starting Grant, Europe’s top research grant for young scientists. His research focuses on Automated Machine Learning with a special focus on Automated Reinforcement Learning, Green AutoML and human-centered AutoML

Tutorial on Bayesian Optimization with SMAC

In this tutorial we introduce you to our hyperparameter optimization (HPO) tool SMAC. SMAC (Sequential Model-Based Algorithm Configuration) [Hutter et al., 2011; Lindauer et al., 2022] is built upon Bayesian Optimization (BO) techniques. It offers a clever intensification routine that efficiently compares different hyperparameter configurations.

During the tutorial, we address the following key aspects:

What is SMAC, our HPO tool? How does it work, and what are its core components?
How do you set up the optimization process using SMAC?
How do we select the most suitable preset configurations for your specific problem?

The first part of the tutorial will explain SMAC’s working principles. The second part will be interactive, where we will guide you through the step-by-step process of using SMAC for HPO. By the end of this tutorial, you will have a solid grasp of SMAC's capabilities and be equipped with the knowledge and practical skills to effectively leverage it for hyperparameter optimization tasks.

Bio. Since 2020 Carolin has been a PhD student at the research group led by Prof. Dr. Marius Lindauer at Leibniz University Hannover which is part of the automl.space. Since then she also enjoyed collaborations with other research groups. Her focus is broadly on Automated Machine Learning, and more explicit on Dynamic Algorithm Configuration and Bayesian Optimization. Apart from the AutoML side, she is also interested in robotics. This might be a remnant of her Bachelor’s and Master’s studies in Mechatronics and Robotics at the Leibniz University Hannover. She is the development lead of SMAC and worked with SMAC for numerous publications. In general, she is driven by the love for automation and making complex algorithms more accessible.

Carolin Benjamins

Phd Student at the Leibniz University Hannover & L3S Research Center

Neutral Benchmarking

If methodological research can be summarized in a single sentence, it is this: „Our new method outperformed the existing ones“. But is it realistic to expect that every new method outperforms existing ones? Do such claims meaningfully help readers and are they trustworthy? What do data analysts need to identify the most appropriate machine learning method for their application? In this talk, I will discuss the importance of neutral method comparison studies as a cornerstone for generating reliable evidence on the performance of methods and providing sound, practical guidance for method selection. Special emphasis will be placed on the design of such studies, the various sources of bias that may affect their results, and the specific considerations associated with automated machine learning in this context.

Bio. Anne-Laure Boulesteix obtained a diploma in engineering from the Ecole Centrale Paris, a diploma in mathematics from the University of Stuttgart (2001) and a PhD in statistics (2005) from the Ludwig Maximilian University (LMU) of Munich. After a postdoc phase in medical statistics, she joined the Medical School of the University of Munich as a junior professor (2009) and professor (2012). She is working at the interface between biostatistics, machine learning and medicine with a particular focus on metascience and evaluation of methods. She is a steering committee member of the STRATOS initiative, founding member of the LMU Open Science Center, former president and current vice-president of the German Region of the International Biometric Society.

Anne-Laure Boulesteix

Professor for Biometry in Molecular Medicine
at LMU Munich

David Holzmüller

Postdoc at INRIA Paris

Lessons from designing better tabular neural networks

Tree-based models have dominated tabular machine learning for a long time. Recently, deep learning models such as our RealMLP have been able to match them on benchmarks. RealMLP extends the classical MLP through a bag of tricks, developed on a meta-train benchmark, and evaluated on a separate meta-test benchmark. In this talk, I want to introduce RealMLP and discuss learnings from tabular model development, including regarding benchmarking, model tuning, and meta-learning.

Bio. David Holzmüller is a postdoctoral researcher at INRIA in Paris, working on machine learning for tabular data and uncertainty quantification in collaboration with the groups of Gaël Varoquaux and Francis Bach. Previously, he obtained his PhD from the University of Stuttgart under the supervision of Ingo Steinwart, exploring topics such as active learning, neural network theory, neural networks for atomistic simulations, and sampling. He owns a M.Sc. and B.Sc. in computer science and a B.Sc. in mathematics.

Wednesday, 11th June 2025

Aaron Klein

Research Group Leader at ScaDS.AI (Center for Scalable Data Analytics and Artificial Intelligence), University of Leipzig

Neural Architecture Search

Neural Architecture Search (NAS) has emerged as a powerful paradigm for automating the design of neural networks, often outperforming manually crafted architectures. This talk will provide an accessible introduction to the fundamentals of neural architectures, followed by an exploration of advanced NAS techniques, with a particular focus on weight sharing approaches that dramatically improve search efficiency. We will conclude by examining practical applications of NAS, including hardware-aware optimization and model compression, highlighting how NAS can tailor models to real-world deployment constraints.

Bio. I am the head of the AutoML research group at ScaDS.AI (Center for Scalable Data Analytics and Artificial Intelligence) in Leipzig. I am also a co-host of the virtual AutoML Seminar as part of the ELLIS units in Berlin and Freiburg. Alongside my collaborators, I lead the development of the open-source library SyneTune for large-scale hyperparameter optimization and neural architecture search.

Until 2024, I worked as an applied scientist at AWS, where I was part of the long-term science team of SageMaker, AWS’s machine learning cloud platform, and the science team of Amazon Q, the GenAI assistant of AWS. Prior to that, I finished my PhD at the University of Freiburg under the supervision of Frank Hutter in 2019. In 2022, my co-authors and I won the best paper award at the AutoML Conference. My collaborators from the University of Freiburg and I won the ChaLearn AutoML Challenge in 2015. I co-organized the workshop on neural architecture search at ICRL 2020 and ICLR 2021, respectively and served as local chair for the AutoML Conference in 2023. I also regularly serve as a reviewer for ICML, ICLR, NeurIPS, TMLR, and JMLR.

Automated Algorithm Design via Large Language Models and Evolutionary Search

Large Language Models (LLMs) are reshaping automated algorithm design, advancing from simple code generation to the implementation of complete optimization algorithms. This talk explores LLM-guided algorithm discovery, with a focus on the LLaMEA framework, an approach that uses evolutionary search to guide LLMs in generating and refining algorithmic codebases. LLaMEA has demonstrated strong performance in discovering competitive metaheuristics and generalizing to other classes, such as Bayesian optimization.
The session will begin with key foundational concepts, including evolutionary computation and benchmarking, and will then walk through how LLaMEA operates, adapts across tasks, and integrates with tools like IOHexperimenter. Attendees will gain both conceptual insight and practical guidance for building, evaluating, and extending LLM-based algorithm design pipelines.

Bio. Elena Raponi is an Assistant Professor in Optimization at the Leiden Institute of Advanced Computer Science (LIACS), where she is a member of the Natural Computing (NaCo) research cluster. Her research lies at the intersection of computer science and engineering, with a focus on black-box optimization, algorithm configuration, and machine learning for industrial applications such as structural mechanics. She has particular expertise in surrogate-based and high-dimensional Bayesian optimization for continuous domains.
Her recent work includes the development of novel methods for automated algorithm design, constrained optimization, and high-dimensional black-box optimization, with applications in automated machine learning (AutoML) and engineering design. She collaborates with industrial partners such as BMW, HRI Europe, Meta AI, Keygene, Stellantis, and Volkswagen Group on applied optimization problems.
Before joining LIACS, she held postdoctoral positions at the School of Engineering and Design at the Technical University of Munich and the Operations Research Group (LIP6) at Sorbonne Université, supported by the prestigious DAAD PRIME fellowship. She holds a Ph.D. in Applied Mathematics from the University of Camerino, Italy.

Elena Raponi

Assistant professor in Bayesian Optimization at the Leiden Institute of Advanced Computer Science (LIACS)

Max Muschalik

PhD student at the Chair of Artificial Intelligence and Machine Learning (AIML) at LMU Munich

shapiq: Explaining Models with Shapley interactions

This talk showcases shapiq, a library for explaining machine learning models through Shapley interaction values. Moving beyond traditional feature attributions, Shapley interactions quantify how combinations of features jointly influence predictions—offering a richer, more faithful view into model behavior. I will give a conceptual overview of interaction-based explanations, demonstrate their relevance in modern AutoML workflows, and highlight shapiq’s ability to work seamlessly with tabular foundation models such as TabPFN. Finally, I will show how everything can be a game—including model design choices—by using shapiq to analyze hyperparameter importance through the lens of cooperative game theory.

Bio. Maximilian Muschalik is a PhD student at the Chair of Artificial Intelligence and Machine Learning (AIML) at LMU Munich, supervised by Prof. Eyke Hüllermeier. His research focuses on explainable artificial intelligence (XAI), particularly on Shapley-based methods for explaining black-box models. He works on efficient algorithms for computing Shapley interactions and builds the Python library shapiq. His broader interests include explanation methods for dynamic learning environments and adaptive models. His research is part of the DFG-funded project TRR 318 Constructing Explainability.

Thursday, 12th June 2025

David Salinas

Senior Research Scientist at the ELLIS Institute in Tübingen and University of Freiburg

Challenges in Evaluating Large Language Models

Large Language Models (LLMs) are increasingly being used in various applications, from basic information retrieval to coding and beyond. As a result, many actors share LLM models, either through blackbox services or open-weight access. Therefore, benchmarking is crucial to determine which model is more suitable for specific use cases.

The generative nature of LLMs, which allows them to generate text, poses a significant challenge in evaluating their performance. A wide range of prompts can be generated, including translation, coding assistance, travel advice, and cooking recipe suggestions. This diversity makes it difficult to assess the effectiveness of LLMs.

In this talk, I will explore various evaluation approaches that have been proposed and discuss their advantages and limitations. In particular, I will focus on LLM-judges, a family of methods that use an LLM to evaluate the free-text generated by another LLM. I will also discuss how AutoML can significantly improve such systems and enable the use of open-weight models instead of closed services like GPT-4.

Bio. David Salinas is a Senior Research Scientist working at the ELLIS Institute in Tübingen and University of Freiburg where he leads a research group to develop fully open European LLMs. Before that, he worked 7 years at Amazon and he has worked intensively in areas such as LLMs, AutoML and forecasting where his work has been published in major conferences and journals in the field. He also has a strong experience delivering ML production systems at AWS and open-source libraries.

AutoML Applications in Language Model Post-Training

Reasoning capabilities represent a critical frontier for large language models (LLMs), but developing them requires extensive proprietary datasets and computational resources. One solution is to use model merging, which offers an alternative by combining the weights, and hence capabilities of multiple models without retraining. However, merging relies on manually-designed strategies for merging hyperparameters, limiting the exploration of potential model combinations and requiring significant human effort. This sounds like an autoML problem - and it is. In this talk I will discuss this problem setting and related applications and a recent publication where we found that we can find good solutions for this application with standard autoML tools adapted to this domain.

Bio. Jonas is a ML researcher in Tübingen, Germany, where he leads the research group for safety- & efficiency- aligned learning. Before this, I’ve spent time at the Universities of Maryland, Siegen and Münster. When it comes to efficient learning, he studies how we can build systems that do more with less, from weight averaging techniques to recursive computation approaches that extend model capabilities. with a particular interest in how these systems reason, and whether we can enhance their reasoning abilities while maintaining efficiency. How do we build mechanisms that let these models learn to be intelligent systems?

Jonas Geiping

Research Group Leader ELLIS Institute & Max-Planck Institute for Intelligent Systems

Jonas
Kübler

Senior Applied Scientist at AWS

Kailash Budhathoki

Senior Researcher at AWS

Inference Optimization of Foundation Models on AI Accelerators

Transformer-based foundation models have revolutionized Generative AI, enabling applications from chatbots to code generation. However, their growing scale brings high inference costs and latency challenges. This talk explores key optimization techniques for efficient inference on AI accelerators like GPUs. Attendees will gain practical insights into deploying large models cost-effectively and at scale.

Bio Kailah. Kailash Budhathoki is a senior researcher at Amazon Web Services (AWS), where he currently leads efforts to optimize foundation models for inference on AI accelerators. Prior to this, he led cross-organizational effort at Amazon to build bias mitigation solutions for Amazon’s first in-house multimodal foundation models. At Amazon Research Lablet Tübingen, he also developed and deployed various causal inference algorithms & tools used across Amazon, which were also open-sourced to the Python DoWhy library. He did his PhD in Computer Science from the Max Planck Institute for Informatics in Saarbrücken.

Bio Jonas. Jonas Kübler is a senior applied scientist at AWS, currently optimizing the inference performance of LLMs. His previous work on statistical testing and quantum machine learning was published at conferences and journals including NeurIPS, ICML, AISTATS and Nature Communications. He did his PhD at the Max Planck Institute for Intelligent Systems in Tübingen.

AutoDS: automating tabular data science

This lecture presents our development of AutoDS, a fully automated AutoML solution for tabular data that currently outperforms AutoGluon on a comprehensive set of Kaggle challenges, achieving an average of 80% on private leaderboards.
The presentation will cover the system architecture and the integration of established techniques including hyperparameter optimization, model blending, data imputation, and feature encoding. I will highlight two key innovations: first, partial hyperopt, a method for efficiently navigating high-dimensional spaces that incorporate feature preprocessor hyperparameters; and second, a novel orchestration agent that makes resource-aware decisions based on the contribution of the models to the blend (ensemble) rather than individual performance metrics.
Additionally, I will describe our systematic protocol for leveraging Kaggle's API in scientific research, which provides a robust benchmarking methodology for tabular AutoML systems. This framework enables reproducible evaluation across diverse real-world datasets.
The lecture will conclude with preliminary results from our ongoing work on performance enhancement using code generation LLMs. These efforts focus on warm-starting hyperparameter optimization and enabling problem-dependent feature engineering, representing promising directions for the next generation of AutoML systems.

Bio. Balázs Kégl is the Head of AI research at Huawei France. He is on leave from the CNRS where he has been a senior research scientist from 2006 and head of the Center for Data Science of the Université Paris-Saclay between 2014 and 2019. Prior to joining the CNRS, he was Assistant Professor at the University of Montreal.

He has published more than two hundred papers on artificial intelligence and its application in particle physics, systems biology, and climate science.

Balázs is co-creator of RAMP (www.ramp.studio), a code-submission platform to accelerate building predictive workflows and to promote collaboration between domain scientists and data scientists.

Balázs Kégl

Head of AI Research,
Huawei France

Jennifer D'Souza

PostDoc in the Data Science and Digital Libraries group at TIB (Leibniz Information Center for Science and Technology University Library).

LLM-assisted Scientific Experimentation: An Overview

In this talk, I will present an overview of how large language models (LLMs) are transforming the landscape of scientific experimentation. Building on advances in AutoML and neural architecture search, LLMs now enable researchers to automate key steps of the experimental workflow—task formulation, implementation, evaluation, and iteration—through natural language interfaces. I will highlight recent developments such as AutoML-GPT, MLcopilot, and MLAgentBench, which leverage LLMs for tasks like hyperparameter tuning and benchmarking. The talk will also explore emerging paradigms involving multi-agent collaboration, tree search, and iterative refinement, signaling a shift toward increasingly autonomous and intelligent experimental systems.

Bio. Jennifer is a senior postdoctoral researcher at the TIB Leibniz Information Centre for Science and Technology, where her expertise in AI, NLP, and scientific knowledge extraction drives her research. At TIB, Jennifer leads the NLP-AI aspect of the Open Research Knowledge Graph (ORKG) project and heads the SCINEXT project, aimed at advancing neuro-symbolic AI and NLP methods for scholarly innovation extraction, supported by the German Federal Ministry of Education and Research.

Friday, 13th June 2025

Anirudh Dagar

Applied Scientist at AWS AI Berlin

AutoGluon Assistant: AutoML in Zero Lines of Code

Can we fully automate machine learning workflows—from raw data to deployable models—using only natural language? In this talk, we explore AutoGluon Assistant’s evolution, an LLM-powered AutoML system that enables no-code model development across diverse data types. Introduced in 2024, it delivered strong results on tabular tasks and placed in the top 10 of the Kaggle AutoML Grand Prix as the only fully automated, no-human-in-the-loop entry—demonstrating its ability to compete with human-led teams. We’ll dive into its core capabilities and highlight recent work on MLZero, a more flexible, multi-agent evolution of the system designed for end-to-end automation of complex, multimodal ML tasks.

Bio. Anirudh Dagar is a Scientist at Amazon Web Services, where he focuses on applied machine learning with an emphasis on AutoML, forecasting, and open-source ML systems. He is a core contributor to several widely-used open-source projects, including AutoGluon, SciPy, and Dive Into Deep Learning (D2L), where he led the development of PyTorch and JAX adaptations. Anirudh holds a Master’s degree from the Indian Institute of Technology Roorkee. He is passionate about ML education and contributes actively to open-source efforts making machine learning more accessible to researchers and practitioners alike.

Unbottling the Secrets of Knowledge Distillation

Despite the advancements in cloud computing, many tasks such as autonomous driving, demand efficient neural networks that can operate seamlessly on embedded devices. This presentation provides a brief overview of various model compression techniques, including quantization, low-rank approximations, pruning, and dynamic sparsity, before focusing on knowledge distillation (KD). KD is the state-of-the-art training method for optimizing model architecture on many tasks. We provide an in-depth understanding of the inner workings of KD, examining its effectiveness, practical applications, and limitations, with a particular emphasis on the amount of compute and data used. Empirical results from ImageNet and smaller datasets highlight the benefits of patient training and appropriate augmentations, even across different architectures. Next, we offer a clear overview of how to design a state-of-the-art pipeline with KD, emphasizing vanilla KD's potential with sufficient compute and optimized hyperparameters.

Bio. Lukas Schott is a Project Lead for Neural Network Compression at Bosch Corporate Research, developing cutting-edge compression methods for foundation models. He holds a PhD in Computer Science from the International Max Planck Research School for Intelligent Systems, focusing on robustness, disentanglement, and generalization in visual representation learning. Schott's expertise spans deep learning, multi-task learning, and computer vision, with publications in top-tier conferences like ICLR, NeurIPS, and ICCV. He is also an active open-source contributor and holds multiple patents.

Lukas Schott

Project Lead at Bosch Corporate Research

Page updated

Report abuse