The Annals of Statistics

Low rank estimation of smooth kernels on graphs

Vladimir Koltchinskii and Pedro Rangel

Full-text: Open access

Abstract

Let $(V,A)$ be a weighted graph with a finite vertex set $V$, with a symmetric matrix of nonnegative weights $A$ and with Laplacian $\Delta$. Let $S_{\ast}: V\times V\mapsto{\mathbb{R}}$ be a symmetric kernel defined on the vertex set $V$. Consider $n$ i.i.d. observations $(X_{j},X_{j}',Y_{j})$, $j=1,\ldots,n$, where $X_{j}$, $X_{j}'$ are independent random vertices sampled from the uniform distribution in $V$ and $Y_{j}\in{\mathbb{R}}$ is a real valued response variable such that ${\mathbb{E}}(Y_{j}|X_{j},X_{j}')=S_{\ast}(X_{j},X_{j}')$, $j=1,\ldots,n$. The goal is to estimate the kernel $S_{\ast}$ based on the data $(X_{1},X_{1}',Y_{1}),\ldots,(X_{n},X_{n}',Y_{n})$ and under the assumption that $S_{\ast}$ is low rank and, at the same time, smooth on the graph (the smoothness being characterized by discrete Sobolev norms defined in terms of the graph Laplacian). We obtain several results for such problems including minimax lower bounds on the $L_{2}$-error and upper bounds for penalized least squares estimators both with nonconvex and with convex penalties.

Article information

Source
Ann. Statist., Volume 41, Number 2 (2013), 604-640.

Dates
First available in Project Euclid: 26 April 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1366980559

Digital Object Identifier
doi:10.1214/13-AOS1088

Mathematical Reviews number (MathSciNet)
MR3099115

Zentralblatt MATH identifier
1360.62272

Subjects
Primary: 62J99: None of the above, but in this section 62H12: Estimation 60B20: Random matrices (probabilistic aspects; for algebraic aspects see 15B52) 60G15: Gaussian processes

Keywords
Matrix completion low-rank matrix estimation optimal error rate minimax error bound matrix Lasso nuclear norm graph Laplacian discrete Sobolev norm

Citation

Koltchinskii, Vladimir; Rangel, Pedro. Low rank estimation of smooth kernels on graphs. Ann. Statist. 41 (2013), no. 2, 604--640. doi:10.1214/13-AOS1088. https://projecteuclid.org/euclid.aos/1366980559


Export citation

References

  • [1] Aubin, J.-P. and Ekeland, I. (1984). Applied Nonlinear Analysis. Wiley, New York.
  • [2] Candès, E. J. and Plan, Y. (2011). Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements. IEEE Trans. Inform. Theory 57 2342–2359.
  • [3] Candès, E. J. and Recht, B. (2009). Exact matrix completion via convex optimization. Found. Comput. Math. 9 717–772.
  • [4] Candès, E. J. and Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inform. Theory 56 2053–2080.
  • [5] de la Peña, V. H. and Giné, E. (1999). Decoupling: From Dependence to Independence. Springer, New York.
  • [6] Gaïffas, S. and Lecué, G. (2011). Hyper-sparse optimal aggregation. J. Mach. Learn. Res. 12 1813–1833.
  • [7] Gross, D. (2011). Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theory 57 1548–1566.
  • [8] Klopp, O. (2012). Noisy low-rank matrix completion with general sampling distribution. Preprint.
  • [9] Koltchinskii, V. (2011). Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems. Lecture Notes in Math. 2033. Springer, Heidelberg.
  • [10] Koltchinskii, V., Lounici, K. and Tsybakov, A. B. (2011). Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. Ann. Statist. 39 2302–2329.
  • [11] Koltchinskii, V. and Rangel, P. (2012). Low rank estimation of similarities on graphs. Preprint.
  • [12] Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin.
  • [13] Negahban, S. and Wainwright, M. J. (2012). Restricted strong convexity and weighted matrix completion: optimal bounds with noise. J. Mach. Learn. Res. 13 1665–1697.
  • [14] Recht, B., Fazel, M. and Parrilo, P. A. (2010). Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev. 52 471–501.
  • [15] Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices. Ann. Statist. 39 887–930.
  • [16] Tropp, J. A. (2012). User-friendly tail bounds for sums of random matrices. Found. Comput. Math. 12 389–434.
  • [17] Tsybakov, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York.