Home

Leonard Papenmeier

PhD Student

Bayesian Optimization, Gaussian Processes, Machine Learning, AutoML

	@leopap@mastodon.social
	LeoIV
	Google Scholar Profile
	LinkedIn
	leonard.papenmeier@cs.lth.se
	University Website

Hello.

Since September 2020, I'm a PhD student at the Wallenberg AI, Autonomous Systems and Software Program and the University of Lund Sweden.

I'm working on the optimization of black-box functions with Bayesian Optimization with a focus on high-dimensional functions with hundreds on input parameters.

I'm interested in exploring the relevance of black-box optimization and developing scalable and reliable algorithms for black-box optimization and applying them to real-world problems.

Research Output.

This paper has been accepted to NeurIPS 2023.

Impactful applications such as materials discovery, hardware design, neural architecture search, or portfolio optimization require optimizing high-dimensional black-box functions with mixed and combinatorial input spaces. While Bayesian optimization has recently made significant progress in solving such problems, an in-depth analysis reveals that the current state-of-the-art methods are not reliable. Their performances degrade substantially when the unknown optima of the function do not have a certain structure. To fill the need for a reliable algorithm for combinatorial and mixed spaces, this paper proposes Bounce that relies on a novel map of various variable types into nested embeddings of increasing dimensionality. Comprehensive experiments show that Bounce reliably achieves and often even improves upon state-of-the-art performance on a variety of high-dimensional problems.

Links:

	Preprint:	https://arxiv.org/abs/2307.00618
	Code:	https://github.com/LeoIV/bounce

Leonard Papenmeier, Luigi Nardi, Matthias Poloczek: Bounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spaces Preprint.

Bayesian optimization is an effective method for optimizing expensive-to-evaluate black-box functions. High-dimensional problems are particularly challenging as the surrogate model of the objective suffers from the curse of dimensionality, which makes accurate modeling difficult. We propose a group testing approach to identify active variables to facilitate efficient optimization in these domains. The proposed algorithm, Group Testing Bayesian Optimization (GTBO), first runs a testing phase where groups of variables are systematically selected and tested on whether they influence the objective. To that end, we extend the well-established theory of group testing to functions of continuous ranges. In the second phase, GTBO guides optimization by placing more importance on the active dimensions. By exploiting the axis-aligned subspace assumption, GTBO is competitive against state-of-the-art methods on several synthetic and real-world high-dimensional optimization tasks. Furthermore, GTBO aids in the discovery of active parameters in applications, thereby enhancing practitioners' understanding of the problem at hand.

Links:

	Code:	https://github.com/gtboauthors/gtbo
	Preprint:	https://arxiv.org/abs/2310.03515

Erik Hellsten*, Carl Hvarfner*, Leonard Papenmeier*, Luigi Nardi High-dimensional Bayesian Optimization with Group Testing Preprint. * Equal contribution.

We present an algorithm for high-dimensional Bayesian Optimization using nested random embeddings (BAxUS). BAxUS starts the optimization in a very low-dimensional sparse embedding (typically only 1-3 dimensions) and increases the embedding as it optimizes. Using the sparse embedding, we can keep previous observations when increasing the embedding throughout the optimization.

Links:

	BoTorch tutorial on BAxUS:	https://botorch.org/tutorials/baxus
	Code:	https://github.com/LeoIV/BAxUS
	Preprint:	https://arxiv.org/abs/2304.11468
	OpenReview:	https://openreview.net/forum?id=e4Wf6112DI
	NeurIPS page (poster + short presentation):	https://neurips.cc/virtual/2022/poster/54175

Leonard Papenmeier, Luigi Nardi, Matthias Poloczek: Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces Advances in Neural Information Processing Systems 35, NeurIPS 2022, New Orleans. OpenReview