Bayesian Analysis

GPU-Accelerated Bayesian Learning and Forecasting in Simultaneous Graphical Dynamic Linear Models

Lutz Gruber and Mike West

Full-text: Open access

Abstract

We discuss Bayesian analysis of dynamic models customized to learning and prediction with increasingly high-dimensional time series. A new framework of simultaneous graphical dynamic models allows the decoupling of analyses into those of a parallel set of univariate time series dynamic models, while flexibly modeling time-varying, cross-series dependencies and volatilities. The strategy allows for exact analysis of univariate time series models that are then coherently linked to represent the full multivariate model. Computation uses importance sampling and variational Bayes ideas, and is ideally suited to GPU-based parallelization. The analysis and its GPU-accelerated implementation is scalable with time series dimension, as we demonstrate in an analysis of a 400-dimensional financial time series.

Article information

Source
Bayesian Anal. Volume 11, Number 1 (2016), 125-149.

Dates
First available in Project Euclid: 2 March 2015

Permanent link to this document
https://projecteuclid.org/euclid.ba/1425304898

Digital Object Identifier
doi:10.1214/15-BA946

Mathematical Reviews number (MathSciNet)
MR3447094

Keywords
decoupling models high-dimensional time series importance sampling parallel computing recoupling models variational Bayes

Citation

Gruber, Lutz; West, Mike. GPU-Accelerated Bayesian Learning and Forecasting in Simultaneous Graphical Dynamic Linear Models. Bayesian Anal. 11 (2016), no. 1, 125--149. doi:10.1214/15-BA946. https://projecteuclid.org/euclid.ba/1425304898


Export citation

References

  • Abramowitz, M. and Stegun, I. A. (1972). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. New York: Dover Publications.
  • Agarwal, D., Chen, D., Lin, L., Shanmugasundaram, J., and Vee, E. (2010). “Forecasting high-dimensional data.” In: Elmagarmid, A. K. and Agrawal, D. (eds.), Proceedings of the ACM SIGMOD International Conference on Management of Data, Indianapolis, USA, 1003–1012. ACM.
  • Aguilar, O. and West, M. (2000). “Bayesian dynamic factor models and portfolio allocation.” Journal of Business & Economic Statistics, 18(3): 338–357.
  • Alspach, D. L. and Sorenson, H. W. (1972). “Nonlinear Bayesian estimation using Gaussian sum approximations.” IEEE Transactions on Automatic Control, 17: 439–448.
  • Anacleto, O., Queen, C., and Albers, C. J. (2013). “Multivariate forecasting of road traffic flows in the presence of heteroscedasticity and measurement errors.” Journal of the Royal Statistical Society: Series C (Applied Statistics), 62: 251–270.
  • Anselin, L. (1988). Spatial econometrics: Methods and models. Dordrecht: Kluwer Academic Publishers.
  • Atkinson, K. (1989). An Introduction to Numerical Analysis. New York: Wiley.
  • Carvalho, C. M. and West, M. (2007). “Dynamic matrix-variate graphical models.” Bayesian Analysis, 2(1): 69–98.
  • Harrison, P. J. and Stevens, C. F. (1971). “A Bayesian approach to short-term forecasting.” Operations Research Quarterly, 22: 341–362.
  • — (1976). “Bayesian forecasting (with discussion).” Journal of the Royal Statistical Society: Series B (Methodological), 38: 205–247.
  • Hosking, J., Natarajan, R., Ghosh, S., Subramanian, S., and Zhang, X. (2013). “Short-term forecasting of the daily load curve for residential electricity usage in the Smart Grid.” Applied Stochastic Models in Business and Industry, 29(6): 604–620.
  • Jaakkola, T. and Jordan, M. I. (2000). “Bayesian parameter estimation via variational methods.” Statistics and Computing, 10: 25–27.
  • Koop, G. (2012). “Using VARs and TVP-VARs with many macroeconomic variables.” Central European Journal of Economic Modelling and Econometrics, 4: 143–167.
  • Lee, A., Yau, C., Giles, M. B., Doucet, A., and Holmes, C. C. (2010). “On the utility of graphics cards to perform massively parallel simulation with advanced Monte Carlo methods.” Journal of Computational & Graphical Statistics, 19(4): 769–789.
  • Lopes, H. F., McCulloch, R. E., and Tsay, R. (2010). “Cholesky stochastic volatility.” Technical report, University of Chicago, Booth Business School.
  • Lopes, H. F. and Polson, N. G. (2010). “Bayesian inference for stochastic volatility.” In: Böcker, K. (ed.), Rethinking Risk Measurement and Reporting: Uncertainty, Bayesian Analysis and Expert Judgement, 515–551. Risk Books/Incisive Financial Publishing Ltd: London.
  • Marsaglia, G. and Tsang, W. W. (2000). “A simple method for generating gamma variables.” ACM Transactions on Mathematical Software, 26(3): 363–372.
  • Mukherjee, C., Kasibhatla, P. S., and West, M. (2014). “Spatially-varying SAR models and Bayesian inference for high-resolution lattice data.” Annals of the Institute of Statistical Mathematics, 66: 473–494.
  • Nakajima, J. and West, M. (2013a). “Bayesian analysis of latent threshold dynamic models.” Journal of Business & Economic Statistics, 31: 151–164.
  • — (2013b). “Bayesian dynamic factor models: Latent threshold approach.” Journal of Financial Econometrics, 11: 116–153.
  • Oliveira, V. D. and Song, J. J. (2008). “Bayesian analysis of simultaneous autoregressive models.” The Indian Journal of Statistics, 70-B: 323–350.
  • Palomo, J., Dunson, D. B., and Bollen, K. (2007). “Bayesian structural equation modeling.” In: Lee, S. Y. (ed.), Handbook of Latent Variables and Related Models, volume 1 of Handbook of Computing and Statistics with Applications, 163–188. North Holland.
  • Pitt, M. and Shephard, N. (1999). “Time varying covariances: A factor stochastic volatility approach (with discussion).” In: Bernardo, J. M., Berger, J. O., Dawid, A. P., and Smith, A. F. M. (eds.), Bayesian Statistics VI, 547–570. Oxford University Press.
  • Prado, R. (2010). “Multi-state models for mental fatigue.” In: O’Hagan, A. and West, M. (eds.), The Handbook of Applied Bayesian Analysis, 845–874. Oxford University Press.
  • Prado, R. and West, M. (2010). Time Series: Modeling, Computation & Inference. Chapman & Hall/CRC Press.
  • Queen, C. M. (1994). “Using the multiregression dynamic model to forecast brand sales in a competitive product market.” Journal of the Royal Statistical Society: Series D (The Statistician), 43: 87–98.
  • Quintana, J. M. and West, M. (1987). “An analysis of international exchange rates using multivariate DLMs.” Journal of the Royal Statistical Society: Series D (The Statistician), 36: 275–281.
  • Smith, A. F. M. and West, M. (1983). “Monitoring renal transplants: An application of the multi-process Kalman filter.” Biometrics, 39: 867–878.
  • Suchard, M. A., Wang, Q., Chan, C., Frelinger, J., Cron, A. J., and West, M. (2010). “Understanding GPU programming for statistical computation: Studies in massively parallel massive mixtures.” Journal of Computational and Graphical Statistics, 19(2): 419–438.
  • Trejo, L. J., Knuth, K., Prado, R., Rosipal, R., Kubitz, K., Kochavi, R., Matthews, B., and Zhang., Y. (2007). “EEG-based estimation of mental fatigue: Convergent evidence for a three-state model.” In: Schmorrow, D. D. and Reeves, L. M. (eds.), Augmented Cognition, HCII 2007, LNAI 4565, 201–211. Berlin Heidelberg: Springer-Verlag.
  • Wand, M. P., Ormerod, J. T., Padoan, S. A., and Fuhrwirth, R. (2011). “Mean field variational Bayes for elaborate distributions.” Bayesian Analysis, 6: 847–900.
  • Wang, H. and West, M. (2009). “Bayesian analysis of matrix normal graphical models.” Biometrika, 96: 821–834.
  • West, M. (1993). “Approximating posterior distributions by mixtures.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 54: 553–568.
  • West, M. and Harrison, P. J. (1989). “Subjective intervention in formal models.” Journal of Forecasting, 8: 33–53.
  • — (1997). Bayesian Forecasting & Dynamic Models. Springer Verlag, 2nd edition.
  • Whittle, P. (1954). “On stationary processes in the plane.” Biometrika, 41: 434–449.
  • Zhao, Z. Y. and West, M. (2014). “Dynamic compositional regression modelling: Application in financial time series forecasting and portfolio decisions.” Technical report, Department of Statistical Science, Duke University.
  • Zhou, X., Nakajima, J., and West, M. (2014). “Bayesian forecasting and portfolio decisions using dynamic dependent sparse factor sparse.” International Journal of Forecasting, 30: 963–980.