Abstract:  Many problems in industry involve objectives that are difficult to write down explicitly: the quality of a generative model's outputs, the relevance of a recommender, the feel of a user interface, or the desirability of a candidate molecule. In these settings, a decision-maker, whether a human expert or another AI system, may struggle to assign numerical scores but can still compare alternatives, for example by choosing the best option from a set or ranking several options from best to worst.

Preferential Bayesian optimization (PBO) studies how to use this kind of comparative feedback for sequential decision-making. Instead of observing function values directly, we place a Gaussian process prior over a latent utility function and update it using preference data. This changes the modeling problem: the surrogate, the likelihood, and the acquisition strategy must all be adapted to observations that take the form of comparisons. The resulting methods retain several features that make Bayesian optimization useful for expensive optimization problems, including sample efficiency and uncertainty-aware query selection.

This tutorial will introduce the main ideas behind PBO and survey how the field has developed. We will discuss how comparison-based feedback changes the usual Bayesian optimization loop, and how these ideas connect to reward modeling and RLHF through shared preference-modeling tools. We will then consider what changes when feedback is expressed in natural language. In that context, we will cover LILO, Language-in-the-Loop Optimization, which uses an LLM as an interface between free-form feedback and structured preference data for a Gaussian process model. The aim is to examine how language can help express richer feedback and prior knowledge while leaving the optimization problem to the Bayesian optimization model.

By the end of the tutorial, attendees should have a clear map of the PBO landscape, a sense of where language models may be useful in this setting and where their role is limited, and a starting point for exploring open questions and available tools.

Abstract: Deploying neural networks on resource-constrained edge devices requires balancing predictive performance with latency, memory, and energy consumption. Neural Architecture Search (NAS) is a promising method towards automating this complex, hardware-dependent design process. In this talk, I will present our recent work on evolutionary NAS for Edge AI, spanning efficient search strategies, architecture analysis, and deployment on microcontrollers. I will discuss how population-based guidance and similarity-aware exploration can reduce search costs while improving robustness and generalization across tasks. I will then introduce hardware-in-the-loop approaches that close the gap between simulated efficiency and real-world performance. Our end-to-end co-design pipelines produce models that are not only accurate but genuinely executable and efficient at the edge.

Abstract: Modern AutoML can, for many purposes, be treated as a dependable engine: given data and a task, it will search for a good model. In industrial practice, though, most of the work of building a useful system lies around that search rather than inside it. Drawing on our experience developing an AutoML system at Beckhoff Automation, this talk takes the search itself as a black box and aims less to present polished solutions than to raise awareness of the problems that surround it once the target is a machine on a factory floor.

A first cluster of challenges comes from the deployment target and the physical world it lives in. Models run inside real-time control systems, where inference latency is a hard constraint whose budget varies enormously from one application to the next. Assumptions that hold on academic image benchmarks -- fixed resolutions, clean and unobstructed views -- often break on the shop floor, where cameras might sit behind cages and safety glass and see parts at awkward scales and aspect ratios. The data is not stationary either: lighting, wear, fouling, and changing materials cause genuine drift over time. And models frequently have to be deployed onto machines that are already running, with little room for downtime.

A second cluster arises from who uses the system and how. Our users are, overwhelmingly, domain experts rather than data scientists, and for those, adjusting the dataset is the option to influence the model quality -- every other decision is abstracted away on purpose. This makes data curation a first-class and surprisingly subtle problem: repeatedly revising the dataset in response to reported scores can quietly overfit the test set, and well-meant user edits can leak information across the train/test split. It also makes communication hard in its own right, since "domain expert" covers a wide spectrum of backgrounds: results have to be mapped into each user's own vocabulary, and the small minority of sophisticated users -- some with a genuine data-science background -- will want more options for interaction themselves, which raises the question of how many levels of engagement, up to an expert mode, a single system should offer.

A third cluster concerns everything required for a trained model to be trusted and to run. Latency must be estimated before deployment rather than measured after it; behaviour must be explained to non-experts in terms they find meaningful; the model must be turned into control code that is both efficient and reliable; and, once deployed, it must be monitored and maintained as the world drifts.

What makes all of these challenges compelling is that none of them are afterthoughts to the search, and yet few are truly settled: most are close relatives of questions this community already studies, reshaped by constraints that benchmarks usually abstract away. I hope to leave the audience with a clearer picture of where these problems lie -- and where academic AutoML research could push the solutions that industrial systems already depend on further than current practice.

Abstract: This talk introduces BoFire, an open-source Python package that combines Bayesian Optimization (BO) with classical design of experiments (DoE) methods for optimizing materials and chemical processes. It will be demonstrated how BoFire can be applied to experimental setups commonly encountered in the chemical industry: First, it will be shown how incorporating ingredient-specific chemical descriptors through mixing rules in the surrogate model can improve the sample-efficiency of formulation optimization. Second, our recent efforts in optimizing gradient programs in liquid chromatography (LC) with respect to both separation quality and measurement run time will be highlighted. In LC, gradient programs are typically expressed as piecewise-linear functions defined by nodes specifying the organic solvent fraction at specific time points, as required by most chromatographic control software. This representation introduces a many-to-one mapping, since different node sequences may encode the same gradient profile. To resolve this, we use Wasserstein distance-based kernels to compare gradient shapes rather than node sequences, ensuring that equivalent gradients are recognized as identical during optimization.

Abstract: Large language models are increasingly used not just to write code, but to search over it — iteratively generating, evaluating, and refining executable programs as a form of optimization. This talk frames automated algorithm design and AutoML as instances of the same underlying problem: optimization in code space, where the search process explores a space of programs rather than a fixed set of hyperparameters. I will introduce LLaMEA, an evolutionary framework that uses LLMs to generate and refine metaheuristic optimization algorithms, and show how the same ideas extend naturally to AutoML pipeline design. I will then present BLADE, a benchmarking framework for LLM-driven algorithm design, and its recent extension to OpenML tasks, including a leakage-safe dataset summarization protocol that prevents LLMs from exploiting memorized benchmark identities. The talk closes with preliminary results comparing LLaMEA and related search methods (EoH, ReEvo, MCTS-AHD) against strong tabular baselines (AutoGluon, TabPFN) on OpenML classification tasks, highlighting both the promise and the current robustness challenges of code-space optimization for AutoML.

Abstract: Finding the best architecture, training data and hyperparameters for foundation modes is complicated by the large amount of compute required to train these models. So how do model developers make these decisions in practice? In this tutorial, I will provide an overview of the strageties that developers and researchers employ to make judgements about behavior at frotier scale. This includes proxy models and scaling laws, observational analysis, and single-run designs that leverage variation within individual training runs. We will discuss what these strategies are and how they work, and also how they can fail. In addition to discussions about model development, this tutorial will also cover strategies that participitiants can employ in their own research. If time permits, we will also discuss hyperparameter transfer, coordinate checks, and the infinite-width theory of scaling up models. By the end of the tutorial, participants will be able to reason about the benefits and limitations of different research strategies for foundation models.

Abstract: Tabular data, spreadsheets organized in rows and columns, are ubiquitous across healthcare, business and finance. The fundamental prediction task of filling in missing values of a label column based on the rest of the columns is essential for thousands of use cases of high societal and commercial value, from early cancer detection to predictive maintenance and demand forecasting. While gradient-boosted decision trees have dominated tabular data for two decades, we demonstrate that this is rapidly changing, with the foundation model revolution having arrived at tabular data. We will show the methods behind this and their extensions to causality, interpretability and robustness, and demo various agentic extensions that achieve the dream of AutoML to democratize machine learning, allowing even non-experts to use state-of-the-art machine learning without writing a single line of code.