Leonard Papenmeier Bayesian Optimization in High Dimensions A Journey Through Subspaces and Challenges

PhD Thesis

This thesis explores the challenges and advancements in high-dimensional Bayesian optimization (HDBO), focusing on understanding, quantifying, and improving optimization techniques in high-dimensional spaces. Bayesian optimization (BO) is a powerful method for optimizing expensive black-box functions, but its effectiveness diminishes as the dimensionality of the search space increases due to the curse of dimensionality. The thesis introduces novel algorithms and methodologies to make HDBO more practical. Key contributions include the development of the BAxUS algorithm, which leverages nested subspaces to optimize high-dimensional problems without estimating the dimensionality of the effective subspace. Additionally, the Bounce algorithm extends these techniques to combinatorial and mixed spaces, providing robust solutions for real-world applications. The thesis also explores the quantification of exploration in acquisition functions, proposing new methods of quantifying exploration and strategies to design more effective optimization approaches. Furthermore, this work analyzes why simple BO setups have recently shown promising performance in high-dimensional spaces, challenging the conventional belief that BO is limited to low-dimensional problems. This thesis offers insights and recommendations for designing more efficient HDBO algorithms by identifying and addressing failure modes such as vanishing gradients and biases in model fitting. Through a combination of theoretical analysis, empirical evaluations, and practical implementations, this thesis contributes to the field of BO by advancing our understanding of high-dimensional optimization and providing actionable methods to improve its performance in complex scenarios

Leonard Papenmeier Semantic Representations in Variational Autoencoders as a Model of the Visual System

Master Thesis

This thesis was written at the Institute of Neural Computation at Ruhr-University Bochum, Germany and supervised by Laurenz Wiskott and Zahra Fayyaz. The goal was to investigate the role of semantic representations in the visual system. I investigated the hypothesis that variational autoencoders (VAEs) learn semantic representations of images by analyzing latent representations of VAEs trained on different datasets. We could not find strong evidence for this hypothesis.

The thesis was graded with 100% (excellent).

Leonard Papenmeier (then: Hövelmann) Sentiment Analysis Based on Word Embeddings: Possible Improvements and Transfer to the German Language

Bachelor Thesis

This thesis was written at Dortmund University of Applied Sciences and Arts, Germany and supervised by Christoph M. Friedrich. I used (then very recent) word embeddings to improve sentiment on German text. I also participated in the GermEval Shared Task on Aspect-based Sentiment in Social Media Customer Feedback (GSCL) 2017 (see publications) and achieved the best result for the German language on one subtask and the second-best result on another subtask.

The thesis was graded with 1.0 (very good).