Speaker Information and Talk Abstracts

Reasoning capabilities represent a critical frontier for large language models (LLMs), but developing them requires extensive proprietary datasets and computational resources. One solution is to use model merging, which offers an alternative by combining the weights, and hence capabilities of multiple models without retraining. However, merging relies on manually-designed strategies for merging hyperparameters, limiting the exploration of potential model combinations and requiring significant human effort. This sounds like an autoML problem - and it is. In this talk I will discuss this problem setting and related applications and a recent publication where we found that we can find good solutions for this application with standard autoML tools adapted to this domain.