Bayesian Analysis

A Vine-copula Based Adaptive MCMC Sampler for Efficient Inference of Dynamical Systems

Daniel Schmidl, Claudia Czado, Sabine Hug, and Fabian J. Theis

Full-text: Open access

Abstract

Statistical inference in high dimensional dynamical systems is often hindered by the unknown dependency structure of model parameters. In particular, the inference of parameterized differential equations (DEs) via Markov chain Monte Carlo (MCMC) samplers often suffers from high proposal rejection rates and is exacerbated by strong autocorrelation structures within the Markov chains leading to poor mixing properties. In this paper, we develop a novel vine-copula based adaptive MCMC approach for efficient parameter inference in dynamical systems with strong parameter interdependence. We exploit the concept of a vine-copula decomposition of distribution densities in order to generate problem-specific proposals for a hybrid independence/random walk Metropolis-Hastings (MH) sampler. The key advantage of this approach is that the corresponding MH proposals generate independent samples from the posterior distribution more efficiently than common competitors. All copula densities can be updated during the sampling procedure for fine-tuning. The performance of our method is assessed on two small-scale examples and finally evaluated on a delay DE model for the JAK2-STAT5 signaling pathway fitted to time-resolved western blot data. We compare our copula-based approach to an independence sampler, a second-order moment-based random walk MH algorithm, and an adaptive MH sampler.

Article information

Source
Bayesian Anal., Volume 8, Number 1 (2013), 1-22.

Dates
First available in Project Euclid: 4 March 2013

Permanent link to this document
https://projecteuclid.org/euclid.ba/1362406647

Digital Object Identifier
doi:10.1214/13-BA801

Mathematical Reviews number (MathSciNet)
MR3036249

Zentralblatt MATH identifier
1329.62144

Keywords
Parameter inference Metropolis-Hastings algorithm independence sampling adaptive MCMC vine copula

Citation

Schmidl, Daniel; Czado, Claudia; Hug, Sabine; Theis, Fabian J. A Vine-copula Based Adaptive MCMC Sampler for Efficient Inference of Dynamical Systems. Bayesian Anal. 8 (2013), no. 1, 1--22. doi:10.1214/13-BA801. https://projecteuclid.org/euclid.ba/1362406647


Export citation

References

  • Aas, K., Czado, C., Frigessi, A., and Bakken, H. (2009). “Pair-copula constructions of multiple dependence.” Insurance, Mathematics and Economics, 44: 182–198.
  • Bayes, M. and Price, M. (1763). “An Essay towards solving a Problem in the Doctrine of Chances. By the late Rev. Mr. Bayes, FRS communicated by Mr. Price, in a letter to John Canton, AMFRS.” Philosophical Transactions (1683-1775), 370–418.
  • Bedford, T. and Cooke, R. (2001). “Probability density decomposition for conditionally dependent random variables modeled by vines.” Annals of Mathematics and Artificial Intelligence, 32: 245–268.
  • — (2002). “Vines - a new graphical model for dependent random variables.” Annals of Statistics, 30(4): 1031–1068.
  • Beichl, I. and Sullivan, F. (2000). “The Metropolis algorithm.” Computing in Science & Engineering, 2(1): 65–69.
  • Brechmann, E. (2010). “Truncated and simplified regular vines and their applications.” Diploma thesis, Technische Universität München.
  • Brechmann, E. and Schepsmeier, U. (2013). “CDVine: Modeling Dependence with C- and D-Vine Copulas in R.” Journal of Statistical Software, 52(3): 1–27.
  • Brooks, S. (1998). “Markov chain Monte Carlo method and its application.” Journal of the Royal Statistical Society: Series D (The Statistician), 47(1): 69–100.
  • Brown, K. and Sethna, J. (2003). “Statistical mechanical approaches to models with many poorly known parameters.” Physical Review E, 68(2): 021904–1–021904–9.
  • Cooper, G. and Hausman, R. (1997). The cell: a molecular approach. Washington DC: ASM Press.
  • Czado, C. (2010). “Pair-copula constructions.” In Jaworki, P., Durante, F., Härdle, W., and Rychlik, T. (eds.), Copula Theory and its Applications, 93–110. Dortrecht (NL): Springer.
  • Dißmann, J., Brechmann, E., Czado, C., and Kurowicka, D. (2013). “Selecting and estimating regular vine copulae and application to financial returns.” Computational Statistics & Data Analysis, 59: 52–69.
  • Gamerman, D. and Lopes, H. (2006). Markov chain Monte Carlo: stochastic simulation for Bayesian inference, volume 68. Boca Raton: Chapman & Hall/CRC.
  • Girolami, M. and Calderhead, B. (2011). “Riemann manifold Langevin and Hamiltonian Monte Carlo methods.” Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2): 123–214.
  • Haario, H., Saksman, E., and Tamminen, J. (2001). “An adaptive Metropolis algorithm.” Bernoulli Journal, 7(2): 223–242.
  • Hastings, W. (1970). “Monte Carlo sampling methods using Markov chains and their applications.” Biometrika, 57(1): 97–109.
  • He, Y., Hodges, J., and Carlin, B. (2007). “Re-considering the variance parameterization in multiple precision models.” Bayesian Analysis, 2(3): 529–556.
  • Hobæk Haff, I. (2013). “Parameter estimation for pair-copula constructions.” Bernoulli Journal, in Press.
  • Hobæk Haff, I., Aas, K., and Frigessi, A. (2010). “On the simplified pair-copula construction–Simply useful or too simplistic?” Journal of Multivariate Analysis, 101(5): 1296–1310.
  • Hoffman, M. and Gelman, A. (2011). “The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo.” arXiv preprint arXiv:1111.4246.
  • Holden, L. (2000). “Convergence of Markov chains in the relative supremum norm.” Journal of Applied Probability, 37(4): 1074–1083.
  • Holden, L., Hauge, R., and Holden, M. (2009). “Adaptive independent Metropolis–Hastings.” The Annals of Applied Probability, 19(1): 395–413.
  • Hu, L. (2006). “Dependence patterns across financial markets: a mixed copula approach.” Applied Financial Economics, 16(10): 717–729.
  • Igaz, P., Toth, S., and Falus, A. (2001). “Biological and clinical significance of the JAK-STAT pathway; lessons from knockout mice.” Inflammation Research, 50(9): 435–441.
  • Joe, H. (1996). “Families of m-variate distributions with given margins and m(m-1)/2 bivariate dependence parameters.” In L. Rüschendorf and B. Schweizer and M. D. Taylor (ed.), Distributions with Fixed Marginals and Related Topics, volume 28 of IMS Lecture Notes – Monograph Series.
  • — (1997). Multivariate Models and Dependence Concepts. London: Chapman & Hall.
  • Kirkpatrick, S., Gelatt, C. D., and Vecchi, M. P. (1983). “Optimization by simulated annealing.” Science, 220(4598): 671–680.
  • Kurowicka, D. and Cooke, R. (2006a). “Completion problem with partial correlation vines.” Linear Algebra and its Applications, 418: 188–200.
  • — (2006b). Uncertainty Analysis with High Dimensional Dependence Modelling. Chichester: Wiley.
  • Kurowicka, D. and Joe, H. (2011). Dependence Modeling - Handbook on Vine Copulae. Singapore: World Scientific Publishing Co.
  • Lawrence, N., Girolami, M., and Rattray, M. (2010). Learning and Inference in Computational Systems Biology. Cambridge: The MIT Press.
  • Li, W., Greiter, M., Oeh, U., and Hoeschen, C. (2011). “Reliability of a new biokinetic model of zirconium in internal dosimetry. Part II. Parameter sensitivity analysis.” Health Physics, 101(6): 677–692.
  • Liu, J. (2008). Monte Carlo Strategies in Scientific Computing. New York: Springer.
  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E., et al. (1953). “Equation of state calculations by fast computing machines.” The Journal of Chemical Physics, 21(6): 1087–1092.
  • Min, A. and Czado, C. (2010). “Bayesian inference for multivariate copulas using pair-copula constructions.” Journal of Financial Econometrics, 8(4): 511–546.
  • — (2011). “Bayesian model selection for multivariate copulas using pair-copula constructions.” Canadian Journal of Statistics, 39: 239–258.
  • Nelsen, R. (2006). An Introduction to Copulas. New York: Springer.
  • Raue, A., Kreutz, C., Maiwald, T., Bachmann, J., Schilling, M., Klingmüller, U., and Timmer, J. (2009). “Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood.” Bioinformatics, 25(15): 1923–1929.
  • Robert, C. and Casella, G. (2004). Monte Carlo statistical methods. New York: Springer.
  • Roberts, G., Gelman, A., and Gilks, W. (1997). “Weak convergence and optimal scaling of random walk Metropolis algorithms.” The Annals of Applied Probability, 7(1): 110–120.
  • Roberts, G. and Rosenthal, J. (2007). “Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms.” Journal of Applied Probability, 44(2): 458–475.
  • Rosenthal, J. (2011). “Optimal proposal distributions and adaptive MCMC.” Handbook of Markov Chain Monte Carlo. Chapman & Hall/CRC Press, Boca Raton, 93–112.
  • Schepsmeier, U., Stoeber, J., and Brechmann, E. (2012). VineCopula: Statistical inference of vine copulas. Technische Universität München, Munich, Germany. URL http://cran.r-project.org/web/packages/VineCopula/index.html
  • Singh, L. and Dattatreya, G. (2006). “Gaussian Mixture Parameter Estimation for Cognitive Radio and Network Surveillance Applications.” WSEAS Transactions on Communications, 5(3): 423–428.
  • Sklar, A. (1959). “Fonctions dé repartition á n dimensions et leurs marges.” Publications de l’Institut de Statistique de l’Université de Paris, 8: 229–231.
  • Smith, M., Min, A., Almeida, C., and Czado, C. (2010). “Modeling longitudinal data using a pair-copula construction decomposition of serial dependence.” Journal of the American Statistical Association, 105: 1467–1479.
  • Subramaniam, P., Torres, B., and Johnson, H. (2001). “So many ligands, so few transcription factors: a new paradigm for signaling through the STAT transcription factors.” Cytokine, 15(4): 175–187.
  • Swameye, I., Müller, T., Timmer, J., Sandra, O., and Klingmüller, U. (2003). “Identification of nucleocytoplasmic cycling as a remote sensor in cellular signaling by databased modeling.” Proceedings of the National Academy of Sciences of the United States of America, 100(3): 1028–1033.
  • Timmer, J., Müller, T., Swameye, I., Sandra, O., and Klingmüller, U. (2004). “Modeling the nonlinear dynamics of cellular signal transduction.” International Journal of Bifurcation and Chaos, 14(6): 2069–2079.
  • Wilkinson, D. (2006). Stochastic Modelling for Systems Biology. Boca Raton: Chapman & Hall/CRC.
  • — (2007). “Bayesian methods in bioinformatics and computational systems biology.” Briefings in Bioinformatics, 8(2): 109–116.

See also

  • Related item: Dawn B. Woodard. Comment on Article by Schmidl et al. Bayesian Anal., Vol. 8, Iss. 1 (2013) 23–26.
  • Related item: Mark Girolami, Antonietta Mira. Comment on Article by Schmidl et al. Bayesian Anal., Vol. 8, Iss. 1 (2013) 27–32.
  • Related item: Daniel Schmidl, Claudia Czado, Sabine Hug, Fabian J. Theis. Rejoinder. Bayesian Anal., Vol. 8, Iss. 1 (2013) 33–42.