Bayesian Analysis

Implicit Copulas from Bayesian Regularized Regression Smoothers

Nadja Klein and Michael Stanley Smith

Full-text: Open access


We show how to extract the implicit copula of a response vector from a Bayesian regularized regression smoother with Gaussian disturbances. The copula can be used to compare smoothers that employ different shrinkage priors and function bases. We illustrate with three popular choices of shrinkage priors—a pairwise prior, the horseshoe prior and a g prior augmented with a point mass as employed for Bayesian variable selection—and both univariate and multivariate function bases. The implicit copulas are high-dimensional, have flexible dependence structures that are far from that of a Gaussian copula, and are unavailable in closed form. However, we show how they can be evaluated by first constructing a Gaussian copula conditional on the regularization parameters, and then integrating over these. Combined with non-parametric margins the regularized smoothers can be used to model the distribution of non-Gaussian univariate responses conditional on the covariates. Efficient Markov chain Monte Carlo schemes for evaluating the copula are given for this case. Using both simulated and real data, we show how such copula smoothing models can improve the quality of resulting function estimates and predictive distributions.

Article information

Bayesian Anal., Volume 14, Number 4 (2019), 1143-1171.

First available in Project Euclid: 20 December 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

distributional regression horseshoe prior penalized splines radial basis regression splines

Creative Commons Attribution 4.0 International License.


Klein, Nadja; Smith, Michael Stanley. Implicit Copulas from Bayesian Regularized Regression Smoothers. Bayesian Anal. 14 (2019), no. 4, 1143--1171. doi:10.1214/18-BA1138.

Export citation


  • Acar, E. F., Craiu, R. V., and Yao, F. (2011). “Dependence calibration in conditional copulas: A nonparametric approach.” Biometrics, 67: 445–453.
  • Belitz, C., Brezger, A., Klein, N., Kneib, T., Lang, S., and Umlauf, N. (2015). “BayesX—Software for Bayesian inference in structured additive regression models. Version 3.0.2.” Available from
  • Carvalho, C. M. and Polson, G., Nicholas (2010). “The horseshoe estimator for sparse signals.” Biometrica, 97: 465–480.
  • Clyde, M. and George, E. I. (2004). “Model uncertainty.” Statistical Science, 19(1): 81–94.
  • Craiu, V. R. and Sabeti, A. (2012). “In mixed company: Bayesian inference for bivariate conditional copula models with discrete and continuous outcomes.” Journal of Multivariate Analysis, 110: 106–120.
  • Demarta, S. and McNeil, A. J. (2005). “The t copula and related copulas.” International Statistical Review, 73(1): 111–129.
  • Fahrmeir, L. and Kneib, T. (2011). Bayesian Smoothing and Regression for Longitudinal, Spatial and Event History Data. New York: Oxford University Press.
  • Fahrmeir, L. and Lang, S. (2001). “Bayesian inference for generalized additive mixed models based on Markov random field priors.” Journal of the Royal Statistical Society. Series C (Applied Statistics), 50: 201–220.
  • Fang, H.-B., Fang, K.-T., and Kotz, S. (2002). “The meta-elliptical distributions with given marginals.” Journal of Multivariate Analysis, 82: 1–16.
  • Grazian, C. and Liseo, B. (2017). “Approximate Bayesian inference in semiparametric copula models.” Bayesian Analysis, 12(4): 991–1016.
  • Harrison, D. and Rubinfeld, D. L. (1978). “Hedonic prices and the demand for clean air.” Journal of Environmental Economics and Management, 5: 81–102.
  • Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. New York/Boca Raton: Chapman & Hall/CRC.
  • Jara, A., Hanson, T., Quintana, F., Müller, P., and Rosner, G. (2011). “DPpackage: Bayesian semi- and nonparametric modeling in R.” Journal of Statistical Software, 40(5): 1–30.
  • Joe, H. (2005). “Asymptotic efficiency of the two-stage estimation method for copula-based models.” Journal of Multivariate Analysis, 94(2): 401–419.
  • Klein, N. and Kneib, T. (2016). “Scale-dependent priors for variance parameters in structured additive distributional regression.” Bayesian Analysis, 11(4): 1071–1106.
  • Klein, N., Kneib, T., Lang, S., and Sohn, A. (2015). “Bayesian structured additive distributional regression with an application to regional income inequality in Germany.” The Annals of Applied Statistics, 9: 1024–1052.
  • Klein, N. and Smith, M. S. (2018). “Supplementary Material containing MATLAB code for “Implicit Copulas from Bayesian Regularized Regression Smoothers”.” Bayesian Analysis. doi:,
  • Lang, S. and Brezger, A. (2004). “Bayesian P-splines.” Journal of Computational and Graphical Statistics, 13: 183–212.
  • Levi, E. and Craiu, R. V. (2016). “Gaussian Process Single Index Models for Conditional Copulas.” arXiv preprint arXiv:1603.03028.
  • Liang, F., Paulo, R., Molina, G., Clyde, M. A., and Berger, J. O. (2008). “Mixtures of g priors for Bayesian variable selection.” Journal of the American Statistical Association, 103(481): 410–423.
  • Murray, J. S., Dunson, D. B., Carin, L., and Lucas, J. E. (2013). “Bayesian Gaussian copula factor models for mixed data.” Journal of the American Statistical Association, 108(502): 656–665.
  • Nelson, R. (2006). An Introduction to Copulas. Springer, 2nd edition.
  • Oh, D. H. and Patton, A. J. (2017). “Modeling dependence in high dimensions with factor copulas.” Journal of Business & Economic Statistics, 35(1): 139–154.
  • Rigby, R. A. and Stasinopoulos, D. M. (2005). “Generalized additive models for location, scale and shape (with discussion).” Journal of the Royal Statistical Society. Series C (Applied Statistics), 54: 507–554.
  • Rue, H. and Held, L. (2005). Gaussian Markov Random Fields. New York/Boca Raton: Chapman & Hall/CRC.
  • Ruppert, D., Wand, M. P., and Carroll, R. J. (2003). Semiparametric Regression. Cambridge University Press.
  • Scheipl, F., Fahrmeir, L., and Kneib, T. (2012). “Spike-and-slab priors for function selection in structured additive regression models.” Journal of the American Statistical Association, 107(500): 1518–1532.
  • Scott, J. G. and Berger, J. O. (2010). “Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem.” The Annals of Statistics, 38(5): 2587–2619.
  • Shimazaki, H. and Shinomoto, S. (2010). “Kernel bandwidth optimization in spike rate estimation.” Journal of Computational Neuroscience, 29(1-2): 171–182.
  • Smith, M. S., Gan, Q., and Kohn, R. (2012). “Modeling dependence using skew t copulas: Bayesian inference and applications.” Journal of Applied Econometrics, 27(3): 500–522.
  • Smith, M. S. and Kohn, R. (1996). “Nonparametric regression using Bayesian variable selection.” Journal of Econometrics, 75(2): 317–343.
  • Smith, M. S. and Kohn, R. (2002). “Parsimonious covariance matrix estimation for longitudinal data.” Journal of the American Statistical Association, 97: 1141–1153.
  • Smith, M. S. and Maneesoonthorn, W. (2018). “Inversion copulas from nonlinear state space models with application to inflation forecasting.” International Journal of Forecasting, 34: 389–407.
  • Smith, M. S. and Vahey, S. (2016). “Asymmetric forecast densities for U.S. macroeconomic variables from a Gaussian copula model of cross-sectional and serial dependence.” Journal of Business and Economic Statistics, 34(3): 416–434.
  • Song, P. (2000). “Multivariate dispersion models generated from Gaussian copula.” Scandinavian Journal of Statistics, 27(2): 305–320.
  • Tipping, M. E. (2001). “Sparse Bayesian learning and the relevance vector machine.” Journal of Machine Learning Research, 1(Jun): 211–244.
  • Wauthier, F. L. and Jordan, M. I. (2010). “Heavy-Tailed Process Priors for Selective Shrinkage.” In Advances in Neural Information Processing Systems, 2406–2414.
  • Wilson, A. G. and Ghahramani, Z. (2010). “Copula Processes.” In Advances in Neural Information Processing Systems, 2460–2468.
  • Wood, S. N., Pya, N., and Säfken, B. (2016). “Smoothing parameter and model selection for general smooth models.” Journal of the American Statistical Association, 111: 1548–1563.

Supplemental materials

  • Supplementary Material for “Implicit Copulas from Bayesian Regularized Regression Smoothers”. This contains extensive additional material organized into five Parts A–F. It includes implementation details, proofs, additional examples, and tables and figures referred to throughout the text.
  • MATLAB code for “Implicit Copulas from Bayesian Regularized Regression Smoothers”. This contains MATLAB files to implement the Bayesian regularized regression smoothers outlined in the paper.