Talk Abstracts

by Jan van Rijn

Automated Machine Learning (AutoML) is a relatively young research area aiming at making high-performance machine learning techniques accessible to a broad set of users. This is achieved by identifying all design choices in creating a machine-learning model and addressing them automatically to generate performance-optimised models. In this opening lecture, we provide an extensive overview of the past and present, as well as future perspectives of AutoML. First, we introduce the concept of AutoML, formally define the problems it aims to solve and describe the three components underlying AutoML approaches: the search space, search strategy and performance evaluation. Next, we discuss hyperparameter optimisation (HPO) techniques commonly used in AutoML systems design, followed by providing an overview of the neural architecture search, a particular case of AutoML for automatically generating deep learning models. We further review and compare available AutoML systems. Finally, we provide a list of open challenges and future research directions.

by Josif Grabocka

This tutorial covers the latest developments in hyperparameter optimization (HPO), with a particular focus on recent advances in pause-and-resume strategies for gray-box and multi-fidelity HPO. Additionally, I will explore the integration of meta- and transfer-learning techniques in HPO, highlighting the use of pre-trained learning curve estimators and end-to-end transformer-based methods. Finally, the tutorial will cover recent advancements in distributed HPO.

by Markus Wagner

Monash University’s Faculty of IT is in the unique position that it offers almost the entire A-to-Z in Sustainable Energy IT: ranging from building control and infrastructure planning to user education and engagement — and all while covering the full spectrum of operational and strategic decision support from individual batteries and EV fleets to precincts and cities (which bring multiple producers and consumers together), and to national level infrastructure. In this session, I will provide an overview of our projects that could benefit from AutoML, as we do many things systematically but manually.

by Oleksandr Shchur 

Time series forecasting is an essential component of decision-making in domains such as energy, retail, and finance. Traditionally, machine learning practitioners have focused on developing task-specific forecasting models that are restricted to a certain dataset or application domain. Inspired by the success of pretrained Large Language Models (LLMs) in natural language processing, it becomes imperative to explore whether a similar approach can be applied to forecasting: Can we train a single large model on huge amounts of diverse time series data, that will generalize to new unseen time series tasks? In this talk, we introduce Chronos, a family of pretrained forecasting models based on minimal modifications to LLM architectures, that accomplishes this goal. Chronos demonstrates remarkable zero-shot performance on unseen datasets, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines. We discuss open challenges in the development and application of pretrained models for time series forecasting, and explore what role AutoML methods can play in overcoming them. 

by Kristian Kersting

This talk will discuss the use of (probabilistic) circuits in automating machine learning tasks, particularly in exploratory data analysis and hyperparameter optimization. Probabilistic circuits (PCs) are models that allow exact and tractable probabilistic inference. Used in the spectral domain, they even offer a unified approach to handling mixed data types. Combined with a mixture model over a dictionary of suitable and interpretable parametric models, they allow one to fully automate exploratory analysis for heterogeneous tabular data at large. Additionally, integrating expert feedback in hyperparameter optimization through circuits allows for more adaptive and responsive machine-learning processes. This presentation will outline the theoretical foundations and practical benefits of these circuits for automating machine learning, supported by empirical evidence.

by Setareh Ariafar

Vizier is the de-facto system for black-box optimization at Google which has tuned more than seventy million objectives across different products and services. The success of Vizier stems from going beyond vanilla Bayesian optimization framework by incorporating user experience, inference speed, flexibility, scalability, and reliability. The purpose of this talk is to showcase a handful of applications where incorporating additional knowledge and design choices such as the non-zero prior mean, the choice of kernel function beyond the typical RBF kernel and effective hyperparameter optimization significantly improves the performance of black-box optimization.

by Katharina Strecker

In numerous scientific disciplines, ML methods offers significant potential for advancing research in novel ways.  However, a considerable challenge for domain experts lies in the necessity for extensive ML experience to effectively apply these methods. Similarly, the Centre for Solar Energy and Hydrogen Research Baden-Württemberg (ZSW) encountered this obstacle when attempting to integrate ML into the work of their researchers.

This talk will demonstrate how the ZSW applied AutoML to create new opportunities for renewable energy domain experts who lacked ML expertise. The session will also illustrate the use of AutoML through a variety of examples in this domain and present the ZSW's self-developed no-code AutoML tool, KI-Lab.EE. 

by Thomas Meißner

Automl is often misunderstood as a replacement for data scientists. This vision falls short to the demand of real wold business applications where machine learning is just a small part of the whole process.

BlueCast is an accelerator rather than a replacement: A lightweight, easy to install library that offers supportive tools for data scientist to make them and their stakeholders happy. EDA, machine learning, model explainability and even uncertainty quantification are all covered. It supports data scientists of all experience levels: Beginners can fire up a whole pipeline in just a few lines of code. Seasoned data scientists enjoy convenience features and have an open API at their hand, allowing the use of custom preprocessing, models and much more. A lightweight and highly flexible swiss army knife for real world data scientists.