Open Access
Translator Disclaimer
March 2023 Inverse Bayesian Optimization: Learning Human Acquisition Functions in an Exploration vs Exploitation Search Task
Nathan Sandholtz, Yohsuke Miyamoto, Luke Bornn, Maurice A. Smith
Author Affiliations +
Bayesian Anal. 18(1): 1-24 (March 2023). DOI: 10.1214/21-BA1303


This paper introduces a probabilistic framework to estimate parameters of an acquisition function given observed human behavior that can be modeled as a collection of sample paths from a Bayesian optimization procedure. The methodology involves defining a likelihood on observed human behavior from an optimization task, where the likelihood is parameterized by a Bayesian optimization subroutine governed by an unknown acquisition function. This structure enables us to make inference on a subject’s acquisition function while allowing their behavior to deviate around the solution to the Bayesian optimization subroutine. To test our methods, we designed a sequential optimization task which forced subjects to balance exploration and exploitation in search of an invisible target location. Applying our proposed methods to the resulting data, we find that many subjects tend to exhibit exploration preferences beyond that of standard acquisition functions to capture. Guided by the model discrepancies, we augment the candidate acquisition functions to yield a superior fit to the human behavior in this task.


We would like to thank Derek Bingham, Nasrin Yousefi, the associate editor, and two anonymous reviewers for helpful comments and suggestions on the paper.


Download Citation

Nathan Sandholtz. Yohsuke Miyamoto. Luke Bornn. Maurice A. Smith. "Inverse Bayesian Optimization: Learning Human Acquisition Functions in an Exploration vs Exploitation Search Task." Bayesian Anal. 18 (1) 1 - 24, March 2023.


Published: March 2023
First available in Project Euclid: 1 February 2022

arXiv: 2104.09237
Digital Object Identifier: 10.1214/21-BA1303

Keywords: Bayesian optimization , directional statistics , exploration vs. exploitation , human cognition , inverse optimization , lab experiment , probabilistic models


Vol.18 • No. 1 • March 2023
Back to Top