Bayesian Analysis
- Bayesian Anal.
- Volume 9, Number 2 (2014), 397-424.
Bayesian Adaptive Smoothing Splines Using Stochastic Differential Equations
Yu Ryan Yue, Daniel Simpson, Finn Lindgren, and Håvard Rue
Full-text: Open access
Abstract
The smoothing spline is one of the most popular curve-fitting methods, partly because of empirical evidence supporting its effectiveness and partly because of its elegant mathematical formulation. However, there are two obstacles that restrict the use of the smoothing spline in practical statistical work. Firstly, it becomes computationally prohibitive for large data sets because the number of basis functions roughly equals the sample size. Secondly, its global smoothing parameter can only provide a constant amount of smoothing, which often results in poor performances when estimating inhomogeneous functions. In this work, we introduce a class of adaptive smoothing spline models that is derived by solving certain stochastic differential equations with finite element methods. The solution extends the smoothing parameter to a continuous data-driven function, which is able to capture the change of the smoothness of the underlying process. The new model is Markovian, which makes Bayesian computation fast. A simulation study and real data example are presented to demonstrate the effectiveness of our method.
Article information
Source
Bayesian Anal. Volume 9, Number 2 (2014), 397-424.
Dates
First available in Project Euclid: 26 May 2014
Permanent link to this document
http://projecteuclid.org/euclid.ba/1401148314
Digital Object Identifier
doi:10.1214/13-BA866
Mathematical Reviews number (MathSciNet)
MR3217001
Keywords
Adaptive smoothing Markov chain Monte Carlo Smoothing spline Stochastic differential equation
Citation
Yue, Yu Ryan; Simpson, Daniel; Lindgren, Finn; Rue, Håvard. Bayesian Adaptive Smoothing Splines Using Stochastic Differential Equations. Bayesian Anal. 9 (2014), no. 2, 397--424. doi:10.1214/13-BA866. http://projecteuclid.org/euclid.ba/1401148314.
References
- Abramovich, F. and Steinberg, D. M. (1996). Improved inference in nonparametric regression using ${L}_{k}$-smoothing splines. Journal of Statistical Planning and Inference 49, 327–341.Mathematical Reviews (MathSciNet): MR1381163
Digital Object Identifier: doi: 10.1016/0378-3758(95)00021-6 - Baladandayuthapani, V., Mallick, B. K. and Carroll, R. J. (2005). Spatially adaptive Bayesian penalized regression splines (P-splines). Journal of Computational and Graphical Statistics 14, 378–394.Mathematical Reviews (MathSciNet): MR2160820
Digital Object Identifier: doi: 10.1198/106186005X47345 - Brezger, A. and Lang, S. (2006). Generalized structured additive regression based on Bayesian P-splines. Computational Statistics and Data Analysis 50, 967–991.Mathematical Reviews (MathSciNet): MR2210741
- Crainiceanu, C., Ruppert, D., Carroll, R., Adarsh, J. and Goodner, B. (2007). Spatially adaptive Penalized splines with heteroscedastic errors. Journal of Computational and Graphical Statistics 16, 265–288.Mathematical Reviews (MathSciNet): MR2370943
Digital Object Identifier: doi: 10.1198/106186007X208768 - Cummins, D. J., Filloon, T. G. and Nychka, D. (2001). Confidence intervals for nonparametric curve estimates: Toward more uniform pointwise coverage. Journal of the American Statistical Association 96, 233–246.Mathematical Reviews (MathSciNet): MR1952734
Digital Object Identifier: doi: 10.1198/016214501750332811 - Duchon, J. (1977). Splines minimizing rotation-invariant semi-norms in Sobolev spaces. In Constructive Theory of Functions of Several Variables (W. Schempp and K. Zeller, eds.), volume 571 of Lecture Notes in Mathematics, 85–100, Springer Berlin / Heidelberg, 10.1007/BFb0086566.
- Eilers, P. and Marx, B. (1996). Flexible smoothing with B-splines and penalties (with discussion). Statistical Science 11, 89–121.Mathematical Reviews (MathSciNet): MR1435485
Digital Object Identifier: doi: 10.1214/ss/1038425655
Project Euclid: euclid.ss/1038425655 - Eilers, P. H. C. and Marx, B. D. (2010). Splines, knots, and penalties. Wiley Interdisciplinary Reviews: Computational Statistics 2, 637–653.
- Eubank, R. L. (1999). Nonparametric Regression and Spline Smoothing. Marcel Dekker Inc.Mathematical Reviews (MathSciNet): MR1680784
- Fahrmeir, L. and Knorr-Held, L. (2000). Dynamic and semiparametric models. In Smoothing and regression: approaches, computation, and application (M. G. Schimek, ed.), 513–544, New York: Wiley.
- Fahrmeir, L. and Lang, S. (2001). Bayesian inference for generalized additive mixed models based on Markov random field priors. Journal of the Royal Statistical Society, Series C: Applied Statistics 50, 201–220.Mathematical Reviews (MathSciNet): MR1833273
Digital Object Identifier: doi: 10.1111/1467-9876.00229 - Fahrmeir, L. and Tutz, G. (2001). Multivariate Statistical Modeling based on Generalized Linear Models. Berlin: Springer.Mathematical Reviews (MathSciNet): MR1832899
- Fahrmeir, L. and Wagenpfeil, S. (1996). Smoothing hazard functions and time-varying effects in discrete duration and competing risks models. Journal of the American Statistical Association 91, 1584–1594.Mathematical Reviews (MathSciNet): MR1439098
Digital Object Identifier: doi: 10.1080/01621459.1996.10476726 - Green, P. J. and Silverman, B. W. (1994). Nonparametric Regression and Generalized Linear Models: a Roughness Penalty Approach. Chapman & Hall Ltd.Mathematical Reviews (MathSciNet): MR1270012
- Gu, C. (2002). Smoothing Spline ANOVA Models. Springer-Verlag Inc, New York.Mathematical Reviews (MathSciNet): MR1876599
- Kimeldorf, G. S. and Wahba, G. (1970). A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Annals of Mathematical Statistics 41, 495–502.Mathematical Reviews (MathSciNet): MR254999
Digital Object Identifier: doi: 10.1214/aoms/1177697089
Project Euclid: euclid.aoms/1177697089 - Krivobokova, T., Crainiceanu, C. M. and Kauermann, G. (2008). Fast Adaptive Penalized Splines. Journal of Computational and Graphical Statistics 17, 1–20.Mathematical Reviews (MathSciNet): MR2424792
Digital Object Identifier: doi: 10.1198/106186008X287328 - Lang, S. and Brezger, A. (2004). Bayesian P-splines. Journal of Computational and Graphical Statistics 13, 183–212.
- Lang, S., Fronk, E. M. and Fahrmeir, L. (2002). Function estimation with locally adaptive dynamic models. Computational Statistics 17, 479–499.
- Lindgren, F. and Rue, H. (2008). On the second-order random walk model for irregular locations. Scandinavian Journal of Statistics 35, 691–700.Mathematical Reviews (MathSciNet): MR2468870
Digital Object Identifier: doi: 10.1111/j.1467-9469.2008.00610.x - Lindgren, F., Rue, H. and Lindström, J. (2011). An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73, 423–498.Mathematical Reviews (MathSciNet): MR2853727
Digital Object Identifier: doi: 10.1111/j.1467-9868.2011.00777.x - Luo, Z. and Wahba, G. (1997). Hybrid adaptive splines. Journal of the American Statistical Association 92, 107–116.Mathematical Reviews (MathSciNet): MR1436101
Digital Object Identifier: doi: 10.1080/01621459.1997.10473607 - O’Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems. Statistical Science 1, 502–527.Mathematical Reviews (MathSciNet): MR874480
Digital Object Identifier: doi: 10.1214/ss/1177013525
Project Euclid: euclid.ss/1177013525 - Pintore, A., Speckman, P. L. and Holmes, C. C. (2006). Spatially adaptive smoothing splines. Biometrika 93, 113–125.Mathematical Reviews (MathSciNet): MR2277744
Digital Object Identifier: doi: 10.1093/biomet/93.1.113 - Rue, H. and Held, L. (2005). Gaussian Markov Random Fields: Theory and Applications, volume 104 of Monographs on Statistics and Applied Probability. Chapman & Hall, London.Mathematical Reviews (MathSciNet): MR2130347
- Rue, H., Martino, S. and Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models using integrated nested Laplace approximations (with discussion). Journal of the Royal Statistical Society, Series B: Statistical Methodology 71, 319–392.Mathematical Reviews (MathSciNet): MR2649602
Digital Object Identifier: doi: 10.1111/j.1467-9868.2008.00700.x - Ruppert, D. and Carroll, R. J. (2000). Spatially-adaptive penalties for spline fitting. Australian & New Zealand Journal of Statistics 42, 205–223.
- Ruppert, D., Wand, M. and Carroll, R. (2003). Semiparametric Regression. Cambridge University Press, Cambridge.Mathematical Reviews (MathSciNet): MR1998720
- Scheipl, F. and Kneib, T. (2009). Locally adaptive Bayesian P-splines with a normal-exponential-gamma prior. Computational Statistics and Data Analysis 53, 3533–3552.Mathematical Reviews (MathSciNet): MR2751944
- Simpson, D., Helton, K. and Lindgren, F. (2012). On the connection between O’Sullivan splines, continuous random walk models, and smoothing splines. Technical report, Norwegian University of Science and Technology.
- Speckman, P. L. and Sun, D. (2003). Fully Bayesian spline smoothing and intrinsic autoregressive priors. Biometrika 90, 289–302.Mathematical Reviews (MathSciNet): MR1986647
Digital Object Identifier: doi: 10.1093/biomet/90.2.289 - Staniswalis, J. G. (1989). Local bandwidth selection for kernel estimates. Journal of the American Statistical Association 84, 284–288.Mathematical Reviews (MathSciNet): MR999690
Digital Object Identifier: doi: 10.1080/01621459.1989.10478767 - Staniswalis, J. G. and Yandell, B. S. (1992). Locally adaptive smoothing splines. Journal of Statistical Computation and Simulation 43, 45–53.Mathematical Reviews (MathSciNet): MR1306183
Digital Object Identifier: doi: 10.1080/00949659208811427 - Wahba, G. (1978). Improper priors, spline smoothing and the problem of guarding against model errors in regression. Journal of the Royal Statistical Society, Series B: Statistical Methodology 40, 364–372.Mathematical Reviews (MathSciNet): MR522220
- Wahba, G. (1990). Spline Models for Observational Data. SIAM [Society for Industrial and Applied Mathematics], Philadelphia.Mathematical Reviews (MathSciNet): MR1045442
- Walsh, J. (1986). An introduction to stochastic partial differential equations. In École d’Été de Probabilités de Saint Flour XIV - 1984 (R. Carmona, H. Kesten and J. Walsh, eds.), volume 1180 of Lecture Notes in Mathematics, 265–439, Springer Berlin / Heidelberg, 10.1007/BFb0074920.Mathematical Reviews (MathSciNet): MR876085
- Wand, M. P. and Ormerod, J. T. (2008). On semiparametric regression with O’Sullivan penalized splines. Australian and New Zealand Journal of Statistics 50, 179–198.Mathematical Reviews (MathSciNet): MR2431193
- Wecker, W. and Ansley, C. (1983). The signal extraction approach to nonlinear regression and spline smoothing. Journal of the American Statistical Association 78, 81–89.Mathematical Reviews (MathSciNet): MR696851
Digital Object Identifier: doi: 10.1080/01621459.1983.10477935 - Wood, S. (2006). Generalized additive models: an introduction with R. CRC Press.Mathematical Reviews (MathSciNet): MR2206355
- Yue, Y., Speckman, P. and Sun, D. (2012). Priors for Bayesian adaptive spline smoothing. Annals of the Institute of Statistical Mathematics 64, 577–613, 10.1007/s10463-010-0321-6.Mathematical Reviews (MathSciNet): MR2880870
Digital Object Identifier: doi: 10.1007/s10463-010-0321-6 - Yue, Y. and Speckman, P. L. (2010). Nonstationary spatial Gaussian Markov random fields. Journal of Computational and Graphical Statistics 19, 96–116.Mathematical Reviews (MathSciNet): MR2654402
Digital Object Identifier: doi: 10.1198/jcgs.2009.08124
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations
Qi, Xin and Zhao, Hongyu, The Annals of Statistics, 2010 - Bayesian adaptive B-spline estimation in proportional hazards frailty models
Sharef, Emmanuel, Strawderman, Robert L., Ruppert, David, Cowen, Mark, and Halasyamani, Lakshmi, Electronic Journal of Statistics, 2010 - Age- and time-varying proportional hazards
models for employment discrimination
Woodworth, George and Kadane, Joseph, The Annals of Applied Statistics, 2010
- Asymptotic efficiency and finite-sample properties of the generalized profiling estimation of parameters in ordinary differential equations
Qi, Xin and Zhao, Hongyu, The Annals of Statistics, 2010 - Bayesian adaptive B-spline estimation in proportional hazards frailty models
Sharef, Emmanuel, Strawderman, Robert L., Ruppert, David, Cowen, Mark, and Halasyamani, Lakshmi, Electronic Journal of Statistics, 2010 - Age- and time-varying proportional hazards
models for employment discrimination
Woodworth, George and Kadane, Joseph, The Annals of Applied Statistics, 2010 - Misinformation in the conjugate prior for the linear model with implications for
free-knot spline modelling
Paciorek, Christopher J., Bayesian Analysis, 2006 - Towards the identification of ordinary differential equations from measurements
J Sprekels, K.-H. Hoffmann, , 1988 - Estimation of constant and time-varying dynamic
parameters of HIV infection in a nonlinear differential equation
model
Liang, Hua, Miao, Hongyu, and Wu, Hulin, The Annals of Applied Statistics, 2010 - Efficient estimation of the partly linear additive Cox
model
Huang, Jian, The Annals of Statistics, 1999 - Spatial models generated by nested stochastic partial differential equations, with an application to global ozone mapping
Bolin, David and Lindgren, Finn, The Annals of Applied Statistics, 2011 - Penalized variable selection procedure for Cox models with semiparametric relative risk
Du, Pang, Ma, Shuangge, and Liang, Hua, The Annals of Statistics, 2010 - Interaction Spline Models and Their Convergence Rates
Chen, Zehua, The Annals of Statistics, 1991