Bayesian Analysis

Bayes Factors for Smoothing Spline ANOVA

Chin-I. Cheng and Paul L. Speckman

Full-text: Open access


This paper describes an approach for variable selection and hypothesis testing in semiparametric additive models using Bayes factors in smoothing spline analysis of variance (SSANOVA) models. Effects can be linear or nonparametric (i.e., smooth or interactions between selected linear and smooth effects). To evaluate the importance of each term in the model, we develop Bayes factors for both linear and nonparametric terms. We compute approximate Bayes factors by Monte Carlo and Laplace integration. These Bayes factors can be computed to compare any two sub-models including one model nested in another. This permits formal tests of any portion or simultaneous portions of an SSANOVA model. We demonstrate this approach with an example.

Article information

Bayesian Anal., Volume 11, Number 4 (2016), 957-975.

First available in Project Euclid: 12 October 2015

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

smoothing spline ANOVA Bayes factor laplace integration reproducing kernel semiparametric model


Cheng, Chin-I.; Speckman, Paul L. Bayes Factors for Smoothing Spline ANOVA. Bayesian Anal. 11 (2016), no. 4, 957--975. doi:10.1214/15-BA974.

Export citation


  • Basu, S. and Chib, S. (2003). “Marginal Likelihood and Bayes Factors for Dirichlet Process Mixture Models.” Journal of the American Statistical Association, 98(461): 224–235.
  • Bowman, A. W. and Azzalini, A. (1997). Applied Smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations. Oxford University Press.
  • Cheng, C.-I. and Speckman, P. L. (2012). “Bayesian Smoothing Spline Analysis of Variance Models.” Computational Statistics and Data Analysis, 56(12): 3945–3958.
  • Chib, S. and Greenberg, E. (2007). “Semiparametric Modeling and Estimation of Instrumental Variable Models.” Journal of Computational and Graphical Statistics, 16(1): 86–114.
  • Chib, S. and Greenberg, E. (2010). “Additive Cubic Spline Regression with Dirichlet Process Mixture Errors.” Journal of Econometrics, 156(2): 322–336.
  • Chib, S. and Jeliazkov, I. (2006). “Inference in Semiparametric Dynamic Models for Binary Longitudinal Data.” Journal of the American Statistical Association, 101(474): 685–700.
  • Cox, D., Koh, E., Wahba, G., and Yandell, B. S. (1988). “Testing the (Parametric) Null Model Hypothesis in (Semiparametric) Partial and Generalized Spline Models.” The Annals of Statistics, 16(1): 113–119.
  • Crainiceanu, C. M. and Ruppert, D. (2004). “Likelihood Ratio Tests in Linear Mixed Models with One Variance Component.” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 66(1): 165–185.
  • Dickey, J. M. (1971). “The weighted likelihood ratio, linear hypotheses on normal location parameters.” The Annals of Mathematical Statistics, 42: 204–223.
  • Eubank, R. L., Hart, J. D., Simpson, D. G., and Stefanski, L. A. (1995). “Testing for additivity in nonparametric regression.” The Annals of Statistics, 23(6): 1896–1920.
  • Fan, J. and Jiang, J. (2005). “Nonparametric Inferences for Additive Models.” Journal of the American Statistical Association, 100(471): 890–907.
  • Fan, J., Zhang, C., and Zhang, J. (2001). “Generalized Likelihood Ratio Statistics and Wilks Phenomenon.” The Annals of Statistics, 29(1): 153–193.
  • Gelman, A., Carlin, J., Stern, H., Dunson, D., Vehtari, A., and Rubin, D. (2014). Bayesian Data Analysis, Third Edition. CRC Press, Boka Raton.
  • George, E. I. (2000). “The Variable Selection Problem.” Journal of the American Statistical Association, 95(452): 1304–1308.
  • Golub, G. H., Heath, M., and Wahba, G. (1979). “Generalized cross-validation as a method for choosing a good ridge parameter.” Technometrics, 21: 215–223.
  • Grizzle, J. E. and Allen, D. M. (1969). “Analysis of growth and dose response curves.” Biometrics, 25: 357–381.
  • Gu, C. (2002). Smoothing Spline ANOVA Models. Springer-Verlag Inc.
  • Guo, W. (2002). “Inference in Smoothing Spline Analysis of Variance.” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 64(4): 887–898.
  • Hart, J. D. (1997). Nonparametric Smoothing and Lack-of-fit Tests. Springer-Verlag Inc.
  • Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall Ltd.
  • Kass, R. E. and Raftery, A. E. (1995). “Bayes Factors.” Journal of the American Statistical Association, 90: 773–795.
  • Kim, M.-K. (2013). “Semi-Parametric Mixed-Effects Models for the Analysis of QT intervals.” PhD, University of California at Santa Barbara.
  • Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O. (2008). “Mixtures of G Priors for Bayesian Variable Selection.” Journal of the American Statistical Association, 103(481): 410–423.
  • Liu, A., Meiring, W., and Wang, Y. (2005). “Testing Generalized Linear Models Using Smoothing Spline Methods.” Statistica Sinica, 15(1): 235–256.
  • Liu, A. and Wang, Y. (2004). “Hypothesis Testing in Smoothing Spline Models.” Journal of Statistical Computation and Simulation, 74(8): 581–597.
  • Lodewyckx, T., Kim, W., Lee, M. D., Tuerlinckx, F., Kuppens, P., and Wagemakers, E.-J. (2011). “A Tutorial on Bayes Factor Estimation with the Product Space Method.” Journal of Mathematical Psychology, 55(5): 331–347.
  • Meng, X.-L. and Wong, W. H. (1996). “Simulating Ratios of Normalizing Constants Via a Simple Identity: A Theoretical Exploration.” Statistica Sinica, 6: 831–860.
  • Min, X. and Sun, D. (2015). “Bayesian model selection for a linear model with grouped covariates.” Annals of the Institute of Statistical Mathematics, 1–27.
  • Morey, R. D., Rouder, J. N., Pratte, M. S., and Speckman, P. L. (2011). “Using MCMC Chain Outputs to Efficiently Estimate Bayes Factors.” Journal of Mathematical Psychology, 55(5): 368–378.
  • Reich, B. J., Storlie, C. B., and Bondell, H. D. (2009). “Variable Selection in Bayesian Smoothing Spline ANOVA Models: Application to Deterministic Computer Codes.” Technometrics, 51(2): 110–120.
  • Rouder, J. N., Morey, R. D., Speckman, P. L., and Province, J. M. (2012). “Default Bayes Factors for ANOVA Designs.” Journal of Mathematical Psychology, 56(5): 356–374.
  • Ruppert, D., Wand, M., and Carroll, R. (2003). Semiparametric Regression. Cambridge University Press, Cambridge.
  • Shi, M. and Dunson, D. B. (2011). “Bayesian Variable Selection Via Particle Stochastic Search.” Statistics & Probability Letters, 81(2): 283–291.
  • Speckman, P. (1988). “Kernel smoothing in partial linear models.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 50(3): 413–436.
  • Spiegelhalter, D. J., Best, N. G., Carlin, B. P., and van der Linde, A. (2002). “Bayesian Measures of Model Complexity and Fit (Pkg: P583-639).” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 64(4): 583–616.
  • Sun, D. and Speckman, P. L. (2008). “Bayesian Hierarchical Linear Mixed Models for Additive Smoothing Splines.” Annals of the Institute of Statistical Mathematics, 60(3): 499–517.
  • Verdinelli, I. and Wasserman, L. (1995). “Computing Bayes factors using a generalization of the Savage–Dickey density ratio.” Journal of the American Statistical Association, 90(430): 614–618.
  • Wahba, G. (1990). Spline Models for Observational Data. SIAM [Society for Industrial and Applied Mathematics].
  • Wahba, G., Wang, Y., Gu, C., Klein, R., and Klein, B. (1995). “Smoothing Spline ANOVA for Exponential Families, with Application to the Wisconsin Epidemiological Study of Diabetic Retinopathy: The 1994 Neyman Memorial Lecture.” The Annals of Statistics, 23(6): 1865–1895.
  • Wang, Y. (2011). Smoothing Splines: Methods and Applications. CRC Press.
  • Wang, Y. and Ke, C. (2004). “ASSIST: A Suite of S functions Implementing Spline smoothing Techniques.”
  • White, G. A. (2006). “Bayesian Semiparametric Spatial And Joint Spatial Temporal Smoothing.” Ph.D. dissertation, University of Missouri Columbia, Department of Statistics.
  • Zellner, A. (1986). “On Assessing Prior Distributions and Bayesian Regression Analysis With g-Prior Distributions.” In: Goel, P. and Zellner, A. (eds.), Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti. Amsterdam: North-Holland/Elsevier.
  • Zellner, A. and Siow, A. (1980). “Posterior Odds Ratios for Selected Regression Hypotheses.” In: Bernardo, J. M., DeGroot, M. H., Lindley, D. V., and Smith, A. F. M. (eds.), Bayesian Statistics: Proceedings of the First International Meeting held in Valencia (Spain), 585–603. University of Valencia.