Bayesian Analysis

Misinformation in the conjugate prior for the linear model with implications for free-knot spline modelling

Christopher J. Paciorek

Abstract

In the conjugate prior for the normal linear model, the prior variance for the coefficients is a multiple of the error variance parameter. However, if the prior mean for the coefficients is poorly chosen, the posterior distribution of the model can be seriously distorted because of prior dependence between the coefficients and error variance. In particular, the error variance will be overestimated, as will the posterior variance of the coefficients. This occurs because the prior mean, which can be thought of as a weighted pseudo-observation, is an outlier with respect to the real observations. While this situation will be easily noticed and avoided in simple models, in more complicated models, the effect can be easily overlooked. The issue arises in the unit information (UI) prior, a conjugate prior in which the prior contributes information equal to that in one observation. In particular, a successful Bayesian nonparametric regression model — Bayesian Adaptive Regression Splines (BARS) — that relies on the UI prior for its model selection step suffers from this problem, and addressing the problem within the Bayesian paradigm alters the penalty on model dimensionality.

Article information

Source
Bayesian Anal., Volume 1, Number 2 (2006), 375-383.

Dates
First available in Project Euclid: 22 June 2012

https://projecteuclid.org/euclid.ba/1340371068

Digital Object Identifier
doi:10.1214/06-BA114

Mathematical Reviews number (MathSciNet)
MR2221270

Zentralblatt MATH identifier
1331.62365

Citation

Paciorek, Christopher J. Misinformation in the conjugate prior for the linear model with implications for free-knot spline modelling. Bayesian Anal. 1 (2006), no. 2, 375--383. doi:10.1214/06-BA114. https://projecteuclid.org/euclid.ba/1340371068