Bayesian Analysis

Sequential Monte Carlo Smoothing with Parameter Estimation

Biao Yang, Jonathan R. Stroud, and Gabriel Huerta

Full-text: Open access

Abstract

We propose two new sequential Monte Carlo (SMC) smoothing methods for general state-space models with unknown parameters. The first is a modification of the particle learning and smoothing (PLS) algorithm of Carvalho, Johannes, Lopes, and Polson (2010), with an adjustment in the backward resampling weights. The second, called Refiltering, is a two-stage method that combines sequential parameter learning and particle smoothing algorithms. We illustrate the methods on three benchmark models using simulated data, and apply them to a stochastic volatility model for daily S&P 500 index returns during the financial crisis. We show that both new methods outperform existing SMC approaches, and that Refiltering is competitive with smoothing approaches based on Markov chain Monte Carlo (MCMC) and Particle MCMC.

Article information

Source
Bayesian Anal., Volume 13, Number 4 (2018), 1137-1161.

Dates
First available in Project Euclid: 29 December 2017

Permanent link to this document
https://projecteuclid.org/euclid.ba/1514516432

Digital Object Identifier
doi:10.1214/17-BA1088

Mathematical Reviews number (MathSciNet)
MR3855366

Keywords
Bayesian smoothing particle filtering particle learning particle smoothing state-space models stochastic volatility

Rights
Creative Commons Attribution 4.0 International License.

Citation

Yang, Biao; Stroud, Jonathan R.; Huerta, Gabriel. Sequential Monte Carlo Smoothing with Parameter Estimation. Bayesian Anal. 13 (2018), no. 4, 1137--1161. doi:10.1214/17-BA1088. https://projecteuclid.org/euclid.ba/1514516432


Export citation

References

  • Andrieu, C., Doucet, A., and Holenstein, R. (2010). “Particle Markov Chain Monte Carlo Methods (with discussion).” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 72(3): 269–342.
  • Briers, M., Doucet, A., and Maskell, S. (2010). “Smoothing Algorithms for State-Space Models.” Annals of the Institute of Statistical Mathematics, 62(1): 61–89.
  • Carlin, B. P., Polson, N. G., and Stoffer, D. S. (1992). “A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling.” Journal of the American Statistical Association, 87(418): 493–500.
  • Carter, C. K. and Kohn, R. (1994). “On Gibbs Sampling for State Space Models.” Biometrika, 81(3): 541–553.
  • Carvalho, C. M., Johannes, M. S., Lopes, H. F., and Polson, N. G. (2010). “Particle Learning and Smoothing.” Statistical Science, 25(1): 88–106.
  • de Jong, P. and Shephard, N. (1995). “The Simulation Smoother for Time Series Models.” Biometrika, 82(2): 339–350.
  • Evensen, G. (1994). “Sequential Data Assimilation with a Nonlinear Quasi-Geostrophic Model using Monte Carlo Methods to Forecast Error Statistics.” Journal of Geophysical Research: Oceans, 99(C5): 10143–10162.
  • Fasiolo, M., Pya, N., and Wood, S. N. (2016). “A Comparison of Inferential Methods for Highly Non-Linear State Space Models in Ecology and Epidemiology.” Statistical Science, 31(1): 96–118.
  • Fearnhead, P. (2002). “Markov Chain Monte Carlo, Sufficient Statistics, and Particle Filters.” Journal of Computational and Graphical Statistics, 11(4): 848–862.
  • Fearnhead, P., Wyncoll, D., and Tawn, J. (2010). “A Sequential Smoothing Algorithm with Linear Computational Cost.” Biometrika, 97(2): 447–464.
  • Frühwirth-Schnatter, S. (1994). “Data Augmentation and Dynamic Linear Models.” Journal of Time Series Analysis, 15(2): 183–202.
  • Gamerman, D. (1998). “Markov Chain Monte Carlo for Dynamic Generalized Linear Models.” Biometrika, 85: 215–227.
  • Geweke, J. and Tanizaki, H. (2001). “Bayesian Estimation of State-Space Models using the Metropolis–Hastings Algorithm within Gibbs Sampling.” Computational Statistics and Data Analysis, 37(2): 151 – 170.
  • Godsill, S. J., Doucet, A., and West, M. (2004). “Monte Carlo Smoothing for Nonlinear Time Series.” Journal of the American Statistical Association, 99(465): 156–168.
  • Gordon, N. J., Salmond, D. J., and Smith, A. F. M. (1993). “Novel Approach to Nonlinear/Non-Gaussian Bayesian State Estimation.” IEE Proceedings F, Radar and Signal Processing, 140(2): 107–113.
  • Jacquier, E., Polson, N. G., and Rossi, P. E. (1994). “Bayesian Analysis of Stochastic Volatility Models.” Journal of Economic and Business Statistics, 12: 371–417.
  • Kantas, N., Doucet, A., Singh, S. S., Maciejowski, J., and Chopin, N. (2015). “On Particle Methods for Parameter Estimation in State-Space Models.” Statistical Science, 30(3): 328–351.
  • Katzfuss, M., Stroud, J. R., and Wikle, C. K. (2016). “Understanding the Ensemble Kalman Filter.” The American Statistician, 70: 350–357.
  • Kim, S., Shephard, N., and Chib, S. (1998). “Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models.” Review of Economic Studies, 65: 361–393.
  • Kitagawa, G. (1996). “Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models.” Journal of Computational and Graphical Statistics, 5(1): 1–25.
  • Kitagawa, G. (1998). “A Self-Organizing State-Space Model.” Journal of the American Statistical Association, 93(443): 1203–1215.
  • Kitagawa, G. and Sato, S. (2001). “Monte Carlo Smoothing and Self-Organising State-Space Model.” In Sequential Monte Carlo Methods in Practice, Statistics for Engineering and Information Science, 177–195. Springer:New York.
  • Liu, J. and West, M. (2001). “Combined Parameter and State Estimation in Simulation-Based Filtering.” In Doucet, A., de Freitas, N., and Gordon, N. (eds.), Sequential Monte Carlo Methods in Practice, Statistics for Engineering and Information Science, 197–223. Springer:New York.
  • Lopes, H. F., Carvalho, C. M., Johannes, M. S., and Polson, N. G. (2011). “Particle Learning for Sequential Bayesian Computation (with discussion).” In Bernardo, J. M., Bayarri, M. J., Berger, J. O., Dawid, A. P., Heckerman, D., Smith, A. F. M., and West, M. (eds.), Bayesian Statistics 9, 317–360. Oxford University Press: Oxford.
  • Lopes, H. F. and Tsay, R. S. (2011). “Particle filters and Bayesian inference in financial econometrics.” Journal of Forecasting, 30(1): 168–209.
  • Newton, M. A. and Raftery, A. E. (1994). “Approximate Bayesian Inference with the Weighted Likelihood Bootstrap (with discussion).” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 56(1): 3–48.
  • Niemi, J. and West, M. (2010). “Adaptive Mixture Modeling Metropolis Methods for Bayesian Analysis of Nonlinear State-Space Models.” Journal of Computational and Graphical Statistics, 19(2): 260–280.
  • Pitt, M. and Shephard, N. (1999). “Filtering via Simulation: Auxiliary Particle Filters.” Journal of the American Statistical Association, 94(446): 590–599.
  • Polson, N. G., Stroud, J. R., and Müller, P. (2008). “Practical Filtering with Sequential Parameter Learning.” Journal of the Royal Statistical Society, Series B (Statistical Methodology), 70(2): 413–428.
  • Scipione, C. M. and Berliner, L. M. (1992). “Bayesian Inference in Nonlinear Dynamical Systems.” Technical report, Ohio State University, Department of Statistics.
  • Scott, S. L. (2002). “Bayesian Methods for Hidden Markov Models: Recursive Computing in the 21st Century.” Journal of the American Statistical Association, 97: 337–351.
  • Shephard, N. and Pitt, M. K. (1997). “Likelihood Analysis of Non-Gaussian Measurement Time Series.” Biometrika, 84: 653–667.
  • Silverman, B. (1986). Density Estimation for Statistics and Data Analysis. Chapman Hall/CRC Press.
  • Snyder, C., Bengtsson, T., Bickel, P., and Anderson, J. (2008). “Obstacles to High-Dimensional Particle Filtering.” Monthly Weather Review, 136: 4629–4640.
  • Storvik, G. (2002). “Particle Filters for State-Space Models with the Presence of Unknown Static Parameters.” IEEE Transactions on Signal Processing, 50(2): 281–289.
  • Stroud, J. R. and Bengtsson, T. (2007). “Sequential State and Variance Estimation within the Ensemble Kalman Filter.” Monthly Weather Review, 135: 3194–3208.
  • Stroud, J. R., Katzfuss, M., and Wikle, C. K. (2017). “A Bayesian Adaptive Ensemble Kalman Filter for Sequential State and Parameter Estimation.” Monthly Weather Review.
  • Stroud, J. R., Müller, P., and Polson, N. G. (2003). “Nonlinear State-Space Models with State-Dependent Variances.” Journal of the American Statistical Association, 98(462): 377–386.
  • Stroud, J. R., Stein, M. L., Lesht, B. M., Schwab, D. J., and Beletsky, D. (2010). “An Ensemble Kalman Filter and Smoother for Satellite Data Assimilation.” Journal of the American Statistical Association, 105: 978–990.
  • Yang, B., Stroud, J. R., and Huerta, G. (2017). “Supplementary Material for Sequential Monte Carlo Smoothing with Parameter Estimation.” Bayesian Analysis.

Supplemental materials

  • Supplementary Material of the Sequential Monte Carlo Smoothing with Parameter Estimation. Supplementary material A provides summaries of the algorithms (MCMC, Particle Filter and PMMH) referenced in the paper. Supplementary material B provides graphical summaries for different estimation methods for the stochastic volatility model with T=3000.