NeurIPS 2025｜CAKE：大模型驱动的贝叶斯优化新配方，让黑箱优化更智能、更高效

Core Insights - The article discusses a new method called Context-Aware Kernel Evolution (CAKE) for Bayesian Optimization, which utilizes large language models (LLMs) to dynamically design optimal Gaussian Process (GP) kernel functions during the optimization process [5][6][14]. Group 1: Methodology - CAKE reimagines the kernel design problem as an "evolutionary process," using LLMs to generate new kernel functions based on existing observational data [17]. - The system maintains a "population" of kernel functions and employs genetic operations such as crossover and mutation to evolve these kernels [19]. - BIC-Acquisition Kernel Ranking (BAKER) is introduced to rank kernel functions based on their model fit and sampling potential, balancing optimization and exploration [21][22]. Group 2: Experimental Results - CAKE was tested against three baseline methods: Fixed (using a single SE or M5 kernel), Adaptive (random selection or BIC selection), and Compositional methods [25]. - In hyperparameter optimization tasks, CAKE achieved the highest final accuracy across all tested machine learning models, demonstrating high sample efficiency, especially in the early stages of optimization [27]. - In dynamic simulation tasks, CAKE outperformed all baseline methods, showing robustness to environmental changes and successfully achieving high scores in challenging tasks [28]. Group 3: Advantages and Future Directions - CAKE offers significant interpretability, allowing for human-readable explanations of kernel structures generated during optimization [34][37]. - The framework is expected to evolve further by incorporating more general kernel function syntax and extending its core ideas to other machine learning tasks, such as SVM and kernel PCA [42].