Open Access
September 2016 Efficient Metropolis–Hastings Proposal Mechanisms for Bayesian Regression Tree Models
Matthew T. Pratola
Bayesian Anal. 11(3): 885-911 (September 2016). DOI: 10.1214/16-BA999


Bayesian regression trees are flexible non-parametric models that are well suited to many modern statistical regression problems. Many such tree models have been proposed, from the simple single-tree model to more complex tree ensembles. Their nonparametric formulation allows one to model datasets exhibiting complex non-linear relationships between the model predictors and observations. However, the mixing behavior of the Markov Chain Monte Carlo (MCMC) sampler is sometimes poor, frequently suffering from local mode stickiness and poor mixing. This is because existing Metropolis–Hastings proposals do not allow for efficient traversal of the model space. We develop novel Metropolis–Hastings proposals that account for the topological structure of regression trees. The first is a novel tree rotation proposal that only requires local changes to the regression tree structure, yet efficiently traverses disparate regions of the model space along contours of high likelihood. The second is a rule perturbation proposal which can be seen as an efficient variation of the change proposal found in existing literature. We implement these samplers and demonstrate their effectiveness on a prediction problem from computer experiments, a test function where structural tree variability is needed to fully explore the posterior and data from a heart rate study.


Download Citation

Matthew T. Pratola. "Efficient Metropolis–Hastings Proposal Mechanisms for Bayesian Regression Tree Models." Bayesian Anal. 11 (3) 885 - 911, September 2016.


Published: September 2016
First available in Project Euclid: 7 March 2016

zbMATH: 1357.62178
MathSciNet: MR3543912
Digital Object Identifier: 10.1214/16-BA999

Keywords: computer experiments , coverage probability , credible interval , Markov chain Monte Carlo , proposal distribution , uncertainty quantification

Rights: Copyright © 2016 International Society for Bayesian Analysis

Vol.11 • No. 3 • September 2016
Back to Top