Electronic Journal of Statistics

Estimating a smooth function on a large graph by Bayesian Laplacian regularisation

Alisa Kirichenko and Harry van Zanten

Full-text: Open access

Abstract

We study a Bayesian approach to estimating a smooth function in the context of regression or classification problems on large graphs. We derive theoretical results that show how asymptotically optimal Bayesian regularisation can be achieved under an asymptotic shape assumption on the underlying graph and a smoothness condition on the target function, both formulated in terms of the graph Laplacian. The priors we study are randomly scaled Gaussians with precision operators involving the Laplacian of the graph.

Article information

Source
Electron. J. Statist. Volume 11, Number 1 (2017), 891-915.

Dates
Received: December 2016
First available in Project Euclid: 28 March 2017

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1490688317

Digital Object Identifier
doi:10.1214/17-EJS1253

Subjects
Primary: 62G20: Asymptotic properties 62C10: Bayesian problems; characterization of Bayes procedures

Keywords
Function estimation on graphs Laplacian regularisation nonparametric Bayes

Rights
Creative Commons Attribution 4.0 International License.

Citation

Kirichenko, Alisa; van Zanten, Harry. Estimating a smooth function on a large graph by Bayesian Laplacian regularisation. Electron. J. Statist. 11 (2017), no. 1, 891--915. doi:10.1214/17-EJS1253. https://projecteuclid.org/euclid.ejs/1490688317


Export citation

References

  • Ando, R. K. and Zhang, T. (2007). Learning on graph with Laplacian regularization., Advances in neural information processing systems 19, 25.
  • Belkin, M., Matveeva, I. and Niyogi, P. (2004). Regularization and semi-supervised learning on large graphs. In, COLT, volume 3120, pp. 624–638. Springer.
  • Borgs, C., Chayes, J. T., Cohn, H. and Zhao, Y. (2014). An $l^p$ theory of sparse graph convergence i: limits, sparse random graph models, and power law distributions., arXiv:1401.2906.
  • Castillo, I., Kerkyacharian, G. and Picard, D. (2014). Thomas Bayes walk on manifolds., Probability Theory and Related Fields 158(3–4), 665–710.
  • Chung, F. (2014). From quasirandom graphs to graph limits and graphlets., Advances in Applied Mathematics 56, 135–174.
  • Cressie, N. (1993)., Statistics for Spatial Data. Wiley.
  • Cvetković, D., Rowlinson, P. and Simić, S. (2010). An introduction to the theory of graph spectra., Cambridge.
  • de Jonge, R. and van Zanten, J. H. (2013). Semiparametric Bernstein–von Mises for the error standard deviation., Electron. J. Stat. 7, 217–243.
  • Dunker, T., Lifshits, M. and Linde, W. (1998). Small deviation probabilities of sums of independent random variables. In, High dimensional probability, pp. 59–74. Springer.
  • Hartog, J. and van Zanten, J. H. (2016). Nonparametric Bayesian label prediction on a graph., ArXiv e-prints.
  • Hein, M. (2006). Uniform convergence of adaptive graph-based regularization. In, International Conference on Computational Learning Theory, pp. 50–64. Springer.
  • Huang, J., Ma, S., Li, H. and Zhang, C.-H. (2011). The sparse Laplacian shrinkage estimator for high-dimensional regression., Annals of statistics 39(4), 2021.
  • Johnson, R. and Zhang, T. (2007). On the effectiveness of Laplacian normalization for graph semi-supervised learning., Journal of Machine Learning Research 8(4).
  • Kirichenko, A. and van Zanten, J. H. (2017). Minimax lower bounds for function estimation on graphs., In preparation.
  • Kolaczyk, E. D. (2009)., Statistical analysis of network data. Springer Series in Statistics. Springer, New York. Methods and models.
  • Li, W. V. and Shao, Q.-M. (2001). Gaussian processes: inequalities, small ball probabilities and applications., Stochastic processes: theory and methods 19, 533–597.
  • Liu, X., Zhao, D., Zhou, J., Gao, W. and Sun, H. (2014). Image interpolation via graph-based Bayesian label propagation., Image Processing, IEEE Transactions on 23(3), 1084–1096.
  • Lovasz, L. (2012)., Large networks and graph limits, volume 60. American Mathematical Soc.
  • Lovász, L. and Szegedy, B. (2006). Limits of dense graph sequences., Journal of Combinatorial Theory, Series B 96(6), 933–957.
  • Mohar, B. (1991a). Eigenvalues, diameter, and mean distance in graphs., Graphs Combin. 7(1), 53–64.
  • Mohar, B. (1991b). The Laplacian spectrum of graphs., Graph theory, combinatorics, and applications 2, 871–898.
  • Rasmussen, C. E. and Williams, C. K. I. (2006). Gaussian processes for machine learning., MIT Press.
  • Rousseau, J. and Szabo, B. (2015). Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator., arXiv preprint arXiv:1504.04814.
  • Sharan, R., Ulitsky, I. and Shamir, R. (2007). Network-based prediction of protein function., Molecular systems biology 3(1), 88.
  • Smola, A. J. and Kondor, R. (2003). Kernels and regularization on graphs. In, Learning theory and kernel machines, pp. 144–158. Springer.
  • Szabó, B., van der Vaart, A. W. and van Zanten, J. H. (2015). Frequentist coverage of adaptive nonparametric Bayesian credible sets., Ann. Statist. 43(4), 1391–1428.
  • van der Vaart, A. W. and van Zanten, J. H. (2008a). Rates of contraction of posterior distributions based on Gaussian process priors., Ann. Statist. 36(3), 1435–1463.
  • van der Vaart, A. W. and van Zanten, J. H. (2008b). Reproducing kernel Hilbert spaces of Gaussian priors., IMS Collections 3, 200–222.
  • Watts, D. J. and Strogatz, S. H. (1998). Collective dynamics of ‘small-world’ networks., Nature 393(6684), 440–442.
  • Wood, D. (1992). The computation of polylogarithms. Technical Report 15-92, University of Kent, Computing Laboratory, University of Kent, Canterbury, UK.
  • Zhu, X. and Ghahramani, Z. (2002). Learning from labeled and unlabeled data with label propagation. Technical, report.
  • Zhu, X., Ghahramani, Z., Lafferty, J. et al. (2003). Semi-supervised learning using gaussian fields and harmonic functions. In, ICML, volume 3, pp. 912–919.