Bayesian Analysis

Scale-Dependent Priors for Variance Parameters in Structured Additive Distributional Regression

Nadja Klein and Thomas Kneib

Full-text: Open access

Enhanced PDF (1114 KB)

Abstract

The selection of appropriate hyperpriors for variance parameters is an important and sensible topic in all kinds of Bayesian regression models involving the specification of (conditionally) Gaussian prior structures where the variance parameters determine a data-driven, adaptive amount of prior variability or precision. We consider the special case of structured additive distributional regression where Gaussian priors are used to enforce specific properties such as smoothness or shrinkage on various effect types combined in predictors for multiple parameters related to the distribution of the response. Relying on a recently proposed class of penalised complexity priors motivated from a general set of construction principles, we derive a hyperprior structure where prior elicitation is facilitated by assumptions on the scaling of the different effect types. The posterior distribution is assessed with an adaptive Markov chain Monte Carlo scheme and conditions for its propriety are studied theoretically. We investigate the new type of scale-dependent priors in simulations and two challenging applications, in particular in comparison to the standard inverse gamma priors but also alternatives such as half-normal, half-Cauchy and proper uniform priors for standard deviations.

Article information

Source
Bayesian Anal., Volume 11, Number 4 (2016), 1071-1106.

Dates
First available in Project Euclid: 24 November 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1448323525

Digital Object Identifier
doi:10.1214/15-BA983

Mathematical Reviews number (MathSciNet)
MR3545474

Zentralblatt MATH identifier
1357.62115

Keywords
Kullback–Leibler divergence Markov chain Monte Carlo simulations penalised complexity prior penalised splines propriety of the posterior

Citation

Klein, Nadja; Kneib, Thomas. Scale-Dependent Priors for Variance Parameters in Structured Additive Distributional Regression. Bayesian Anal. 11 (2016), no. 4, 1071--1106. doi:10.1214/15-BA983. https://projecteuclid.org/euclid.ba/1448323525


Export citation

References

  • Bayarri, M. J. and García-Donato, G. (2008). “Generalization of Jeffreys divergence-based priors for Bayesian hypothesis testing.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 70: 981–1003.
  • Belitz, C., Brezger, A., Klein, N., Kneib, T., Lang, S. and Umlauf, N. (2015). “BayesX – Software for Bayesian inference in structured additive regression models.” Version 3.0.2. Available from http://www.bayesx.org.
  • Berger, J. O. (2006). “The case for objective Bayesian analysis (with discussion).” Bayesian Analysis 1: 385–402.
  • Berger, J. O., Bernardo, J. M. and Sun, D. (2009). “The formal definition of reference priors.” The Annals of Statistics 37: 905–938.
  • Bernardo, J. M. (1979). “Reference posterior distributions for Bayesian inference.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 41: 113–147.
  • Brezger, A. and Lang, S. (2006). “Generalized structured additive regression based on Bayesian P-splines.” Computational Statistics & Data Analysis 50: 967–991.
  • Eilers, P. H. and Marx, B. D. (1996). “Flexible smoothing using B-splines and penalized likelihood.” Statistical Science 11: 89–121.
    Mathematical Reviews (MathSciNet): MR1435485
    Digital Object Identifier: doi:10.1214/ss/1038425655
    Project Euclid: euclid.ss/1038425655
  • Fahrmeir, L. and Kneib, T. (2009). “Propriety of posteriors in structured additive regression models: Theory and empirical evidence.” Journal of Statistical Planning and Inference 139: 843–859.
  • Fahrmeir, L., Kneib, T. and Lang, S. (2004). “Penalized structured additive regression for space-time data: A Bayesian perspective.” Statistica Sinica 14: 731–761.
  • Fahrmeir, L., Kneib, T., Lang, S. and Marx, B. (2013). Regression – Models, Methods and Applications. Springer, Berlin.
  • Fahrmeir, L. and Lang, S. (2001). “Bayesian semiparametric regression analysis of multicategorical time–space data.” Annals of the Institute of Statistical Mathematics 53: 11–30.
    Mathematical Reviews (MathSciNet): MR1820949
    Digital Object Identifier: doi:10.1023/A:1017904118167
  • Frühwirth-Schnatter, S. and Wagner, H. (2010). “Stochastic model specification search for Gaussian and partial non-Gaussian state space models.” Journal of Econometrics 154: 85–100.
  • Frühwirth-Schnatter, S. and Wagner, H. (2011). “Bayesian variable selection for random intercept modeling of Gaussian and non-Gaussian data.” In: J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith and M. West (eds), Bayesian Statistics 9, Oxford, pp. 165–200.
  • Gamerman, D. (1997). “Sampling from the posterior distribution in generalized linear mixed models.” Statistics and Computing 7: 57–68.
  • García-Donato, G. and Sun, D. (2007). “Objective priors for hypothesis testing in one-way random effects models.” The Canadian Journal of Statistics 35: 303–320.
  • Gelman, A. (2005). “Analysis of variance: why it is more important than ever (with discussion).” The Annals of Statistics 33: 1–53.
  • Gelman, A. (2006). “Prior distributions for variance parameters in hierarchical models.” Bayesian Analysis 1: 515–533.
  • Gelman, A., Jakulin, A., Pittau, M. G. and Su, Y. S. (2008). “A weakly informative default prior distribution for logistic and other regression models.” The Annals of Applied Statistics 2: 1360–1383.
  • George, A. and Liu, J. W. (1981). Computer Solution of Large Sparse Positive Definite Systems. Prentice-Hall, Englewood Cliffs.
  • George, E. and Mc Culloch, R. (1993). “Variable selection via Gibbs sampling,.” Journal of the American Statistical Association 88: 881–889.
  • Ghosh, M. (2011). “Objective priors: An introduction for frequentists (with discussion).” Statistical Science 26: 187–202.
  • Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman & Hall/CRC, New York/Boca Raton.
  • Hastings, W. K. (1970). “Monte Carlo sampling methods using Markov chains and their applications.” Biometrika 57: 97–109.
  • Hodges, J. S. (2013). Richly Parameterized Linear Models: Additive, Time Series, and Spatial Models Using Random Effects. Chapman & Hall/CRC.
  • Ishwaran, H. and Rao, S. (2005). “Spike and slab variable selection: frequentist and Bayesian strategies.” Annals of Statistics 33: 730–773.
  • Jeffreys, H. (1961). Theory of Probability. Oxford University Press, Oxford.
  • Jerak, A. and Wagner, S. (2006). “Modeling probabilities of patent oppositions in a Bayesian semiparametric regression framework.” Empirical Economics 31: 513–533.
  • Kammann, E. E. and Wand, M. P. (2003). “Geoadditive models.” Journal of the Royal Statistical Society: Series C (Applied Statistics) 52: 1–18.
  • Klein, N. (2015). sdPrior: Scale-Dependent Hyperpriors in Structured Additive Distributional Regression. R package version 0.3.
  • Klein, N. and Kneib, T. (2015). “Scale-dependent priors for variance parameters in structured additive distributional regression: Supplement.” Bayesian Analysis.
  • Klein, N., Kneib, T., Klasen, S. and Lang, S. (2015). “Bayesian structured additive distributional regression for multivariate responses.” Journal of the Royal Statistical Society: Series C (Applied Statistics) 64: 569–591.
  • Klein, N., Kneib, T. and Lang, S. (2015). “Bayesian generalized additive models for location, scale and shape for zero-inflated and overdispersed count data.” Journal of the American Statistical Association 110: 405–419.
  • Klein, N., Kneib, T., Lang, S. and Sohn, A. (2015). “Bayesian structured additive distributional regression with an application to regional income inequality in germany.” Annals of Applied Statistics 9: 1024–1052.
  • Kneib, T., Hothorn, T. and Tutz, G. (2009). “Variable selection and model choice in geoadditive regression models.” Biometrics 65: 626–634.
  • Krivobokova, T., Kneib, T. and Claeskens, G. (2010). “Simultaneous confidence bands for penalized spline estimators.” Journal of the American Statistical Association 105: 852–863.
  • Lang, S. and Brezger, A. (2004). “Bayesian P-splines.” Journal of Computational and Graphical Statistics 13: 183–212.
    Mathematical Reviews (MathSciNet): MR2044877
    Digital Object Identifier: doi:10.1198/1061860043010
  • Lang, S., Umlauf, N., Wechselberger, P., Harttgen, K. and Kneib, T. (2014). “Multilevel structured additive regression.” Statistics and Computing 24: 223–238.
  • Lindgren, F., Rue, H. and Lindström, J. (2011). “An explicit link between Gaussian fields and Gaussian Markov random fields: The SPDE approach (with discussion).” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73: 423–498.
  • Polson, N. G. and Scott, J. G. (2012). “On the half-Cauchy prior for a global scale parameter.” Bayesian Analysis 7: 887–902.
  • Rigby, R. A. and Stasinopoulos, D. M. (2005). “Generalized additive models for location, scale and shape (with discussion).” Journal of the Royal Statistical Society: Series C (Applied Statistics) 54: 507–554.
  • Rue, H. (2001). “Fast sampling of Gaussian Markov random fields with applications.” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63: 325–338.
  • Rue, H. and Held, L. (2005). Gaussian Markov Random Fields. Chapman & Hall/CRC, New York/Boca Raton.
  • Rue, H., Martino, S. and Chopin, N. (2009). “Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion).” Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71: 319–392.
  • Ruppert, D., Wand, M. P. and Carroll, R. J. (2003). Semiparametric Regression. Cambridge University Press.
  • Scheipl, F., Fahrmeir, L. and Kneib, T. (2012). “Spike-and-slab priors for function selection in structured additive regression models.” Journal of the American Statistical Association 107: 1518–1532.
  • Simpson, D., Rue, H. Martins, T. G., Riebler, A. and Sørbye, S. H. (2014). Penalising model component complexity: A principled, practical approach to constructing priors, arXiv:1403.4630, Norwegian University of Sciences and Technology, Trondheim, Norway, submitted to Statistical Science.
    arXiv: 1403.4630
  • Sørbye, S. H. and Rue, H. (2014). “Scaling intrinsic Gaussian Markov random field priors in spatial modelling.” Spatial Statistics 8: 39–51.
  • Sun, D., Tsutakawa, R. K. and He, Z. (2001). Propriety of posteriors with improper priors in hierarchical linear mixed models.” Statistica Sinica 11: 77–95.
  • Wand, M. P. (2000). “A comparison of regression spline smoothing procedures.” Computational Statistics 15: 443–462.
  • Wood, S. N. (2006). Generalized Additive Models: An Introduction with R, Chapman & Hall/CRC, New York/Boca Raton.

Supplemental materials

International Society for Bayesian Analysis

Bayesian Analysis

New content alerts

Email RSS ToC RSS Article
  • You have access to this content.
  • You have partial access to this content.
  • You do not have access to this content.
Turn Off MathJax

What is MathJax?

More like this

+ See more