Hello.
I successfully defended my PhD thesis on June 12th, and I'm open to new opportunities - preferably in the Netherlands or Germany and either in academia or industry
I'm also open to remote work.
Here's my
CV. If you are interested in working with me, please reach out via
email or
LinkedIn.
I'm Leonard Papenmeier, a machine learning researcher currently based in Lund, Sweden.
From September 2020 to June 2024, I obtained my PhD from the University of Lund, Sweden and the Wallenberg AI, Autonomous Systems and Software Program (WASP) under the supervision of Luigi Nardi.
Before that, I completed my Master's degree in Applied Computer Science at the Ruhr-University Bochum, Germany and obtained a Bachelor's degree in Software Engineering from the University of Applied Sciences in Dortmund, Germany.
I'm working on the optimization of black-box functions with Bayesian Optimization with a focus on
high-dimensional functions with hundreds on input parameters.I'm interested in exploring the limits of high-dimensional Bayesian optimization and
developing scalable and reliable algorithms for optimizing a broad set of high-dimensional problems.
Research Output.
Recent work reported that simple Bayesian optimization methods perform well for high-dimensional real-world tasks, seemingly contradicting prior work and tribal knowledge.
This paper investigates the 'why'.
We identify fundamental challenges that arise in high-dimensional Bayesian optimization and explain why recent methods succeed. Our analysis shows that vanishing gradients caused by Gaussian process initialization schemes play a major role in the failures of high-dimensional Bayesian optimization and that methods that promote local search behaviors are better suited for the task.
We find that maximum likelihood estimation of Gaussian process length scales suffices for state-of-the-art performance. Based on this, we propose a simple variant of maximum likelihood estimation called MSR that leverages these findings to achieve state-of-the-art performance on a comprehensive set of real-world applications.
We also present targeted experiments to illustrate and confirm our findings.
A well-balanced exploration-exploitation trade-off is crucial for successful acquisition functions in Bayesian optimization.
However, there is a lack of quantitative measures for exploration, making it difficult to analyze and compare different acquisition functions.
This work introduces two novel approaches - observation traveling salesman distance and observation entropy - to quantify the exploration characteristics of acquisition functions based on their selected observations.
Using these measures, we examine the explorative nature of several well-known acquisition functions across a diverse set of black-box problems, uncover links between exploration and empirical performance, and reveal new relationships among existing acquisition functions.
Beyond enabling a deeper understanding of acquisition functions, these measures also provide a foundation for guiding their design in a more principled and systematic manner.
Bayesian optimization is a widely used method for optimizing expensive black-box functions, with Expected Improvement being one of the most commonly used acquisition functions.
In contrast, information-theoretic acquisition functions aim to reduce uncertainty about the function's optimum and are often considered fundamentally distinct from EI. In this work, we challenge this prevailing perspective by introducing a unified theoretical framework, Variational Entropy Search, which reveals that EI and information-theoretic acquisition functions are more closely related than previously recognized.
We demonstrate that EI can be interpreted as a variational inference approximation of the popular information-theoretic acquisition function, named Max-value Entropy Search. Building on this insight, we propose VES-Gamma, a novel acquisition function that balances the strengths of EI and MES.
Extensive empirical evaluations across both low- and high-dimensional synthetic and real-world benchmarks demonstrate that VES-Gamma is competitive with state-of-the-art acquisition functions and in many cases outperforms EI and MES.
Impactful applications such as materials discovery, hardware design, neural architecture search, or
portfolio optimization require optimizing high-dimensional black-box functions with mixed and combinatorial
input spaces.
While Bayesian optimization has recently made significant progress in solving such problems, an in-depth
analysis reveals that the current state-of-the-art methods are not reliable.
Their performances degrade substantially when the unknown optima of the function do not have a certain
structure.
To fill the need for a reliable algorithm for combinatorial and mixed spaces, this paper proposes Bounce
that relies on a novel map of various variable types into nested embeddings of increasing dimensionality.
Comprehensive experiments show that Bounce reliably achieves and often even improves upon state-of-the-art
performance on a variety of high-dimensional problems.
Bayesian optimization is an effective method for optimizing expensive-to-evaluate black-box functions.
High-dimensional problems are particularly challenging as the surrogate model of the objective suffers from
the curse of dimensionality, which makes accurate modeling difficult.
We propose a group testing approach to identify active variables to facilitate efficient optimization in these domains.
The proposed algorithm, Group Testing Bayesian Optimization (GTBO), first runs a testing phase where groups
of variables are systematically selected and tested on whether they influence the objective.
To that end, we extend the well-established theory of group testing to functions of continuous ranges.
In the second phase, GTBO guides optimization by placing more importance on the active dimensions.
By exploiting the axis-aligned subspace assumption, GTBO is competitive against state-of-the-art methods on
several synthetic and real-world high-dimensional optimization tasks.
Furthermore, GTBO aids in the discovery of active parameters in applications, thereby enhancing
practitioners'
understanding of the problem at hand.
We present an algorithm for high-dimensional Bayesian Optimization using nested random embeddings
(BAxUS).
BAxUS starts the optimization in a very low-dimensional sparse embedding (typically only 1-3
dimensions)
and
increases the embedding as it optimizes.
Using the sparse embedding, we can keep previous observations when increasing the embedding
throughout
the
optimization.
We present Bencher, a modular benchmarking framework for black-box optimization that fundamentally decouples benchmark execution from optimization logic. Unlike prior suites that focus on combining many benchmarks in a single project, Bencher introduces a clean abstraction boundary: each benchmark is isolated in its own virtual Python environment and accessed via a unified, version-agnostic remote procedure call (RPC) interface. This design eliminates dependency conflicts and simplifies the integration of diverse, real-world benchmarks, which often have complex and conflicting software requirements. Bencher can be deployed locally or remotely via Docker or on high-performance computing (HPC) clusters via Singularity, providing a containerized, reproducible runtime for any benchmark. Its lightweight client requires minimal setup and supports drop-in evaluation of 80 benchmarks across continuous, categorical, and binary domains.
Leonard Papenmeier