Bernoulli

  • Bernoulli
  • Volume 21, Number 4 (2015), 2073-2092.

Some remarks on MCMC estimation of spectra of integral operators

Radosław Adamczak and Witold Bednorz

Full-text: Open access

Abstract

We prove a law of large numbers for empirical approximations of the spectrum of a kernel integral operator by the spectrum of random matrices based on a sample drawn from a Markov chain, which complements the results by V. Koltchinskii and E. Giné for i.i.d. sequences. In a special case of Mercer’s kernels and geometrically ergodic chains, we also provide exponential inequalities, quantifying the speed of convergence.

Article information

Source
Bernoulli, Volume 21, Number 4 (2015), 2073-2092.

Dates
Received: November 2013
First available in Project Euclid: 5 August 2015

Permanent link to this document
https://projecteuclid.org/euclid.bj/1438777586

Digital Object Identifier
doi:10.3150/14-BEJ635

Mathematical Reviews number (MathSciNet)
MR3378459

Zentralblatt MATH identifier
1350.60027

Keywords
approximation of spectra kernel operators MCMC algorithms random matrices

Citation

Adamczak, Radosław; Bednorz, Witold. Some remarks on MCMC estimation of spectra of integral operators. Bernoulli 21 (2015), no. 4, 2073--2092. doi:10.3150/14-BEJ635. https://projecteuclid.org/euclid.bj/1438777586


Export citation

References

  • [1] Aaronson, J., Burton, R., Dehling, H., Gilat, D., Hill, T. and Weiss, B. (1996). Strong laws for $L$- and $U$-statistics. Trans. Amer. Math. Soc. 348 2845–2866.
  • [2] Adamczak, R. (2008). A tail inequality for suprema of unbounded empirical processes with applications to Markov chains. Electron. J. Probab. 13 1000–1034.
  • [3] Adamczak, R. and Bednorz, W. (2013). Exponential concentration inequalities for additive functionals of Markov chains. Available at arXiv:1201.3569.
  • [4] Andrieu, C., Jasra, A., Doucet, A. and Del Moral, P. (2011). On nonlinear Markov chain Monte Carlo. Bernoulli 17 987–1014.
  • [5] Arcones, M.A. (1998). The law of large numbers for $U$-statistics under absolute regularity. Electron. Commun. Probab. 3 13–19 (electronic).
  • [6] Athreya, K.B. and Ney, P. (1978). A new approach to the limit theory of recurrent Markov chains. Trans. Amer. Math. Soc. 245 493–501.
  • [7] Baxendale, P.H. (2005). Renewal theory and computable convergence rates for geometrically ergodic Markov chains. Ann. Appl. Probab. 15 700–738.
  • [8] Bertail, P. and Clémençon, S. (2011). A renewal approach to Markovian $U$-statistics. Math. Methods Statist. 20 79–105.
  • [9] Bhatia, R. and Elsner, L. (1994). The Hoffman–Wielandt inequality in infinite dimensions. Proc. Indian Acad. Sci. Math. Sci. 104 483–494.
  • [10] Chen, X. (1999). Limit theorems for functionals of ergodic Markov chains with general state space. Mem. Amer. Math. Soc. 139 xiv+203.
  • [11] Douc, R., Fort, G., Moulines, E. and Soulier, P. (2004). Practical drift conditions for subgeometric rates of convergence. Ann. Appl. Probab. 14 1353–1377.
  • [12] Douc, R., Guillin, A. and Moulines, E. (2008). Bounds on regeneration times and limit theorems for subgeometric Markov chains. Ann. Inst. H. Poincaré Probab. Statist. 44 239–257.
  • [13] Horn, R.A. and Johnson, C.R. (2013). Matrix Analysis, 2nd ed. Cambridge: Cambridge Univ. Press.
  • [14] Koltchinskii, V. and Giné, E. (2000). Random matrix approximation of spectra of integral operators. Bernoulli 6 113–167.
  • [15] Mendelson, S. and Pajor, A. (2005). Ellipsoid approximation using random vectors. In Learning Theory. Lecture Notes in Computer Science 3559 429–443. Berlin: Springer.
  • [16] Mendelson, S. and Pajor, A. (2006). On singular values of matrices with independent rows. Bernoulli 12 761–773.
  • [17] Meyn, S. and Tweedie, R.L. (2009). Markov Chains and Stochastic Stability, 2nd ed. Cambridge: Cambridge Univ. Press.
  • [18] Minh, H.Q., Niyogi, P. and Yao, Y. (2006). Mercer’s theorem, feature maps, and smoothing. In Learning Theory. Lecture Notes in Computer Science 4005 154–168. Berlin: Springer.
  • [19] Montgomery-Smith, S.J. (1993). Comparison of sums of independent identically distributed random vectors. Probab. Math. Statist. 14 281–285 (1994).
  • [20] Nummelin, E. (1978). A splitting technique for Harris recurrent Markov chains. Z. Wahrsch. Verw. Gebiete 43 309–318.
  • [21] Nummelin, E. (1984). General Irreducible Markov Chains and Nonnegative Operators. Cambridge Tracts in Mathematics 83. Cambridge: Cambridge Univ. Press.
  • [22] Roberts, G.O. and Rosenthal, J.S. (2004). General state space Markov chains and MCMC algorithms. Probab. Surv. 1 20–71.
  • [23] Rosasco, L., Belkin, M. and De Vito, E. (2010). On learning with integral operators. J. Mach. Learn. Res. 11 905–934.
  • [24] Shawe-Taylor, J., Williams, C.K.I., Cristianini, N. and Kandola, J. (2005). On the eigenspectrum of the Gram matrix and the generalization error of kernel-PCA. IEEE Trans. Inform. Theory 51 2510–2522.
  • [25] Smale, S. and Zhou, D.-X. (2009). Geometry on probability spaces. Constr. Approx. 30 311–323.
  • [26] Sun, H. (2005). Mercer theorem for RKHS on noncompact sets. J. Complexity 21 337–349.
  • [27] Vershynin, R. (2000). On large random almost Euclidean bases. Acta Math. Univ. Comenian. (N.S.) 69 137–144.
  • [28] von Luxburg, U., Belkin, M. and Bousquet, O. (2008). Consistency of spectral clustering. Ann. Statist. 36 555–586.