Bayesian prediction with adaptive ridge estimators

Abstract

The Bayesian linear model framework has become an increasingly popular building block in regression problems. It has been shown to produce models with good predictive power and can be used with basis functions that are nonlinear in the data to provide flexible estimated regression functions. Further, model uncertainty can be accounted for by Bayesian model averaging. We propose a simpler way to account for model uncertainty that is based on generalized ridge regression estimators. This is shown to predict well and to be much more computationally efficient than standard model averaging methods. Further, we demonstrate how to efficiently mix over different sets of basis functions, letting the data determine which are most appropriate for the problem at hand.