Misinformation in the conjugate prior for the linear model with implications for free-knot spline modelling

Abstract

In the conjugate prior for the normal linear model, the prior variance for the coefficients is a multiple of the error variance parameter. However, if the prior mean for the coefficients is poorly chosen, the posterior distribution of the model can be seriously distorted because of prior dependence between the coefficients and error variance. In particular, the error variance will be overestimated, as will the posterior variance of the coefficients. This occurs because the prior mean, which can be thought of as a weighted pseudo-observation, is an outlier with respect to the real observations. While this situation will be easily noticed and avoided in simple models, in more complicated models, the effect can be easily overlooked. The issue arises in the unit information (UI) prior, a conjugate prior in which the prior contributes information equal to that in one observation. In particular, a successful Bayesian nonparametric regression model — Bayesian Adaptive Regression Splines (BARS) — that relies on the UI prior for its model selection step suffers from this problem, and addressing the problem within the Bayesian paradigm alters the penalty on model dimensionality.