The Annals of Statistics
- Ann. Statist.
- Volume 41, Number 1 (2013), 91-124.
Convergence rate of Markov chain methods for genomic motif discovery
Dawn B. Woodard and Jeffrey S. Rosenthal
Full-text: Open access
Abstract
We analyze the convergence rate of a simplified version of a popular Gibbs sampling method used for statistical discovery of gene regulatory binding motifs in DNA sequences. This sampler satisfies a very strong form of ergodicity (uniform). However, we show that, due to multimodality of the posterior distribution, the rate of convergence often decreases exponentially as a function of the length of the DNA sequence. Specifically, we show that this occurs whenever there is more than one true repeating pattern in the data. In practice there are typically multiple such patterns in biological data, the goal being to detect the most well-conserved and frequently-occurring of these. Our findings match empirical results, in which the motif-discovery Gibbs sampler has exhibited such poor convergence that it is used only for finding modes of the posterior distribution (candidate motifs) rather than for obtaining samples from that distribution. Ours are some of the first meaningful bounds on the convergence rate of a Markov chain method for sampling from a multimodal posterior distribution, as a function of statistical quantities like the number of observations.
Article information
Source
Ann. Statist., Volume 41, Number 1 (2013), 91-124.
Dates
First available in Project Euclid: 5 March 2013
Permanent link to this document
https://projecteuclid.org/euclid.aos/1362493041
Digital Object Identifier
doi:10.1214/12-AOS1075
Mathematical Reviews number (MathSciNet)
MR3059411
Zentralblatt MATH identifier
1347.62048
Subjects
Primary: 62F15: Bayesian inference
Secondary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Keywords
Gibbs sampler DNA slow mixing spectral gap multimodal
Citation
Woodard, Dawn B.; Rosenthal, Jeffrey S. Convergence rate of Markov chain methods for genomic motif discovery. Ann. Statist. 41 (2013), no. 1, 91--124. doi:10.1214/12-AOS1075. https://projecteuclid.org/euclid.aos/1362493041
References
- Andrieu, C., Doucet, A. and Holenstein, R. (2010). Particle Markov chain Monte Carlo methods (with discussion). J. R. Stat. Soc. Ser. B Stat. Methodol. 72 269–342.Mathematical Reviews (MathSciNet): MR2758115
Digital Object Identifier: doi:10.1111/j.1467-9868.2009.00736.x - Belloni, A. and Chernozhukov, V. (2009). On the computational complexity of MCMC-based estimators in large samples. Ann. Statist. 37 2011–2055.Mathematical Reviews (MathSciNet): MR2533478
Zentralblatt MATH: 1175.65015
Digital Object Identifier: doi:10.1214/08-AOS634
Project Euclid: euclid.aos/1245332839 - Berk, R. H. (1966). Limiting behavior of posterior distributions when the model is incorrect. Ann. Math. Statist. 37 51–58.Mathematical Reviews (MathSciNet): MR189176
Zentralblatt MATH: 0151.23802
Digital Object Identifier: doi:10.1214/aoms/1177699477
Project Euclid: euclid.aoms/1177699476 - Bhatnagar, N. and Randall, D. (2004). Torpid mixing of simulated tempering on the Potts model. In Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms 478–487. ACM, New York.Mathematical Reviews (MathSciNet): MR2291087
- Borgs, C., Chayes, J. T., Frieze, A., Kim, J. H., Tetali, P., Vigoda, E. and Vu, V. H. (1999). Torpid mixing of some MCMC algorithms in statistical physics. In Proceedings of the 40th IEEE Symposium on Foundations of Computer Science 218–229. IEEE, New York.Mathematical Reviews (MathSciNet): MR1917562
- Del Moral, P., Doucet, A. and Jasra, A. (2006). Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 411–436.Mathematical Reviews (MathSciNet): MR2278333
Zentralblatt MATH: 1105.62034
Digital Object Identifier: doi:10.1111/j.1467-9868.2006.00553.x - Diaconis, P. and Saloff-Coste, L. (1993). Comparison theorems for reversible Markov chains. Ann. Appl. Probab. 3 696–730.Mathematical Reviews (MathSciNet): MR1233621
Zentralblatt MATH: 0799.60058
Digital Object Identifier: doi:10.1214/aoap/1177005359
Project Euclid: euclid.aoap/1177005359 - Diaconis, P. and Saloff-Coste, L. (1996). Logarithmic Sobolev inequalities for finite Markov chains. Ann. Appl. Probab. 6 695–750.Mathematical Reviews (MathSciNet): MR1410112
Zentralblatt MATH: 0867.60043
Digital Object Identifier: doi:10.1214/aoap/1034968224
Project Euclid: euclid.aoap/1034968224 - Diaconis, P. and Stroock, D. (1991). Geometric bounds for eigenvalues of Markov chains. Ann. Appl. Probab. 1 36–61.Mathematical Reviews (MathSciNet): MR1097463
Zentralblatt MATH: 0731.60061
Digital Object Identifier: doi:10.1214/aoap/1177005980
Project Euclid: euclid.aoap/1177005980 - Fort, G., Moulines, E., Roberts, G. O. and Rosenthal, J. S. (2003). On the geometric ergodicity of hybrid samplers. J. Appl. Probab. 40 123–146.Mathematical Reviews (MathSciNet): MR1953771
Zentralblatt MATH: 1028.65002
Digital Object Identifier: doi:10.1239/jap/1044476831
Project Euclid: euclid.jap/1044476831 - Gelman, A. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statist. Sci. 7 457–472.
- Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern. Anal. Mach. Intell. 6 721–741.
- Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. In Bayesian Statistics, 4 (PeñíScola, 1991) (J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith, eds.) 169–193. Oxford Univ. Press, New York.Mathematical Reviews (MathSciNet): MR1380276
- Green, P. J. and Richardson, S. (2002). Hidden Markov models and disease mapping. J. Amer. Statist. Assoc. 97 1055–1070.Mathematical Reviews (MathSciNet): MR1951259
Zentralblatt MATH: 1046.62117
Digital Object Identifier: doi:10.1198/016214502388618870 - Hans, C., Dobra, A. and West, M. (2007). Shotgun stochastic search for “large $p$” regression. J. Amer. Statist. Assoc. 102 507–516.Mathematical Reviews (MathSciNet): MR2370849
Digital Object Identifier: doi:10.1198/016214507000000121 - Jarner, S. F. and Hansen, E. (2000). Geometric ergodicity of Metropolis algorithms. Stochastic Process. Appl. 85 341–361.Mathematical Reviews (MathSciNet): MR1731030
Zentralblatt MATH: 0997.60070
Digital Object Identifier: doi:10.1016/S0304-4149(99)00082-4 - Jensen, S. T., Liu, X. S., Zhou, Q. and Liu, J. S. (2004). Computational discovery of gene regulatory binding motifs: A Bayesian perspective. Statist. Sci. 19 188–204.Mathematical Reviews (MathSciNet): MR2082154
Digital Object Identifier: doi:10.1214/088342304000000107
Project Euclid: euclid.ss/1089808282 - Johnson, A. A. and Jones, G. L. (2010). Gibbs sampling for a Bayesian hierarchical general linear model. Electron. J. Stat. 4 313–333.Mathematical Reviews (MathSciNet): MR2645487
Digital Object Identifier: doi:10.1214/09-EJS515
Project Euclid: euclid.ejs/1268655652 - Jones, G. L. and Hobert, J. P. (2001). Honest exploration of intractable probability distributions via Markov chain Monte Carlo. Statist. Sci. 16 312–334.Mathematical Reviews (MathSciNet): MR1888447
Digital Object Identifier: doi:10.1214/ss/1015346317
Project Euclid: euclid.ss/1015346317 - Jones, G. L. and Hobert, J. P. (2004). Sufficient burn-in for Gibbs samplers for a hierarchical random effects model. Ann. Statist. 32 784–817.Mathematical Reviews (MathSciNet): MR2060178
Zentralblatt MATH: 1048.62069
Digital Object Identifier: doi:10.1214/009053604000000184
Project Euclid: euclid.aos/1083178947 - Kamatani, K. (2011). Weak consistency of Markov chain Monte Carlo methods. Technical report. Available at http://arxiv.org/abs/1103.5679.
- Kellis, M., Patterson, N., Birren, B., Berger, B. and Lander, E. S. (2004). Methods in comparative genomics: Genome correspondence, gene identification and regulatory motif discovery. J. Comput. Biol. 11 319–355.
- Kullback, S. (1959). Information Theory and Statistics. Wiley, New York.Mathematical Reviews (MathSciNet): MR103557
- Lawrence, C. E., Altschul, S. F., Boguski, M. S., Liu, J. S., Neuwald, A. F. and Wootton, J. C. (1993). Detecting subtle sequence signals: A Gibbs sampling strategy for multiple alignment. Science 262 208–214.
- Liang, F. and Wong, W. H. (2000). Evolutionary Monte Carlo: Applications to $C_p$ model sampling and change point problem. Statist. Sinica 10 317–342.
- Liu, J. S. (1994). The collapsed Gibbs sampler in Bayesian computations with applications to a gene regulation problem. J. Amer. Statist. Assoc. 89 958–966.Mathematical Reviews (MathSciNet): MR1294740
Zentralblatt MATH: 0804.62033
Digital Object Identifier: doi:10.1080/01621459.1994.10476829 - Liu, X., Brutlag, D. L. and Liu, J. S. (2001). BioProspector: Discovering conserved DNA motifs in upstream regulatory regions of co-expressed genes. Pacific Symposium on Biocomputing 6 127–138.
- Liu, J. S., Neuwald, A. F. and Lawrence, C. E. (1995). Bayesian models for multiple local sequence alignment and Gibbs sampling strategies. J. Amer. Statist. Assoc. 90 1156–1170.
- Liu, J. S., Wong, W. H. and Kong, A. (1995). Covariance structure and convergence rate of the Gibbs sampler with various scans. J. Roy. Statist. Soc. Ser. B 57 157–169.Mathematical Reviews (MathSciNet): MR1325382
- Madras, N. and Randall, D. (2002). Markov chain decomposition for convergence rate analysis. Ann. Appl. Probab. 12 581–606.Mathematical Reviews (MathSciNet): MR1910641
Zentralblatt MATH: 1017.60080
Digital Object Identifier: doi:10.1214/aoap/1026915617
Project Euclid: euclid.aoap/1026915617 - Madras, N. and Zheng, Z. (2003). On the swapping algorithm. Random Structures Algorithms 22 66–97.Mathematical Reviews (MathSciNet): MR1943860
- Mira, A. (2001). Ordering and improving the performance of Monte Carlo Markov chains. Statist. Sci. 16 340–350.Mathematical Reviews (MathSciNet): MR1888449
Digital Object Identifier: doi:10.1214/ss/1015346319
Project Euclid: euclid.ss/1015346319 - Mossel, E. and Vigoda, E. (2006). Limitations of Markov chain Monte Carlo algorithms for Bayesian inference of phylogeny. Ann. Appl. Probab. 16 2215–2234.Mathematical Reviews (MathSciNet): MR2288719
Zentralblatt MATH: 1121.60078
Digital Object Identifier: doi:10.1214/105051600000000538
Project Euclid: euclid.aoap/1169065222 - Neuwald, A. F., Liu, J. S. and Lawrence, C. E. (1995). Gibbs motif sampling: Detection of bacterial outer membrane protein repeats. Protein Sci. 4 1618–1632.
- Peskun, P. H. (1973). Optimum Monte-Carlo sampling using Markov chains. Biometrika 60 607–612.Mathematical Reviews (MathSciNet): MR362823
Zentralblatt MATH: 0271.62041
Digital Object Identifier: doi:10.1093/biomet/60.3.607 - Roberts, G. O. and Rosenthal, J. S. (2004). General state space Markov chains and MCMC algorithms. Probab. Surv. 1 20–71.Mathematical Reviews (MathSciNet): MR2095565
Zentralblatt MATH: 1189.60131
Digital Object Identifier: doi:10.1214/154957804100000024
Project Euclid: euclid.ps/1099928648 - Roberts, G. O. and Sahu, S. K. (2001). Approximate predetermined convergence properties of the Gibbs sampler. J. Comput. Graph. Statist. 10 216–229.Mathematical Reviews (MathSciNet): MR1939698
Digital Object Identifier: doi:10.1198/10618600152627915 - Rosenthal, J. S. (1995). Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90 558–566.Mathematical Reviews (MathSciNet): MR1340509
Zentralblatt MATH: 0824.60077
Digital Object Identifier: doi:10.1080/01621459.1995.10476548 - Rosenthal, J. S. (1996). Analysis of the Gibbs sampler for a model related to James–Stein estimators. Statist. Comput. 6 269–275.
- Roth, F. P., Hughes, J. D., Estep, P. W. and Church, G. M. (1998). Finding DNA regulatory motifs within unaligned noncoding sequences clustered by whole-genome mRNA quantitation. Nat. Biotechnol. 16 939–945.
- Sinclair, A. (1992). Improved bounds for mixing rates of Markov chains and multicommodity flow. Combin. Probab. Comput. 1 351–370.Mathematical Reviews (MathSciNet): MR1211324
Zentralblatt MATH: 0801.90039
Digital Object Identifier: doi:10.1017/S0963548300000390 - Tierney, L. (1998). A note on Metropolis–Hastings kernels for general state spaces. Ann. Appl. Probab. 8 1–9.Mathematical Reviews (MathSciNet): MR1620401
Zentralblatt MATH: 0935.60053
Digital Object Identifier: doi:10.1214/aoap/1027961031
Project Euclid: euclid.aoap/1027961031 - Woodard, D. B. and Rosenthal, J. S. (2013). Supplement to “Convergence rate of Markov chain methods for genomic motif discovery.” DOI:10.1214/12-AOS1075SUPP.
- Woodard, D. B., Schmidler, S. C. and Huber, M. (2009a). Conditions for rapid mixing of parallel and simulated tempering on multimodal distributions. Ann. Appl. Probab. 19 617–640.Mathematical Reviews (MathSciNet): MR2521882
Zentralblatt MATH: 1171.65008
Digital Object Identifier: doi:10.1214/08-AAP555
Project Euclid: euclid.aoap/1241702244 - Woodard, D. B., Schmidler, S. C. and Huber, M. (2009b). Sufficient conditions for torpid mixing of parallel and simulated tempering. Electron. J. Probab. 14 780–804.Mathematical Reviews (MathSciNet): MR2495560
Zentralblatt MATH: 1189.65021
Digital Object Identifier: doi:10.1214/EJP.v14-638
Supplemental materials
- Supplementary material: Supplemental article. Provides additional proofs.Digital Object Identifier: doi:10.1214/12-AOS1075SUPP
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Computational Discovery of Gene Regulatory Binding Motifs: A Bayesian Perspective
Jensen, Shane T., Liu, X. Shirley, Zhou, Qing, and Liu, Jun S., Statistical Science, 2004 - Coupling hidden Markov models for the discovery
of Cis-regulatory modules in multiple species
Zhou, Qing and Wong, Wing Hung, The Annals of Applied Statistics, 2007 - Survival analysis of DNA mutation motifs with penalized proportional hazards
Feng, Jean, Shaw, David A., Minin, Vladimir N., Simon, Noah, and Matsen IV, Frederick A., The Annals of Applied Statistics, 2019
- Computational Discovery of Gene Regulatory Binding Motifs: A Bayesian Perspective
Jensen, Shane T., Liu, X. Shirley, Zhou, Qing, and Liu, Jun S., Statistical Science, 2004 - Coupling hidden Markov models for the discovery
of Cis-regulatory modules in multiple species
Zhou, Qing and Wong, Wing Hung, The Annals of Applied Statistics, 2007 - Survival analysis of DNA mutation motifs with penalized proportional hazards
Feng, Jean, Shaw, David A., Minin, Vladimir N., Simon, Noah, and Matsen IV, Frederick A., The Annals of Applied Statistics, 2019 - A Vine-copula Based Adaptive MCMC Sampler for Efficient Inference of Dynamical Systems
Schmidl, Daniel, Czado, Claudia, Hug, Sabine, and Theis, Fabian J., Bayesian Analysis, 2013 - Bayesian Models and Gibbs Sampling Strategies for Local Graph Alignment and Motif Identification in Stochastic Biological Networks
Jiang, Rui, Chen, Ting, and Sun, Fengzhu, Communications in Information & Systems, 2009 - Accelerated Gibbs sampling of normal distributions using matrix splittings and polynomials
Fox, Colin and Parker, Albert, Bernoulli, 2017 - Markov Chains for Exploring Posterior Distributions
Tierney, Luke, The Annals of Statistics, 1994 - Gibbs sampling for a Bayesian hierarchical general linear model
Johnson, Alicia A. and Jones, Galin L., Electronic Journal of Statistics, 2010 - Bayesian joint modeling of multiple gene
networks and diverse genomic data to identify target genes of a
transcription factor
Wei, Peng and Pan, Wei, The Annals of Applied Statistics, 2012 - Trace class Markov chains for the Normal-Gamma Bayesian shrinkage model
Zhang, Liyuan, Khare, Kshitij, and Xing, Zeren, Electronic Journal of Statistics, 2019