Open Access
December 2014 Minimax risk of matrix denoising by singular value thresholding
David Donoho, Matan Gavish
Ann. Statist. 42(6): 2413-2440 (December 2014). DOI: 10.1214/14-AOS1257


An unknown $m$ by $n$ matrix $X_{0}$ is to be estimated from noisy measurements $Y=X_{0}+Z$, where the noise matrix $Z$ has i.i.d. Gaussian entries. A popular matrix denoising scheme solves the nuclear norm penalization problem $\operatorname{min}_{X}\|Y-X\|_{F}^{2}/2+\lambda\|X\|_{*}$, where $\|X\|_{*}$ denotes the nuclear norm (sum of singular values). This is the analog, for matrices, of $\ell_{1}$ penalization in the vector case. It has been empirically observed that if $X_{0}$ has low rank, it may be recovered quite accurately from the noisy measurement $Y$.

In a proportional growth framework where the rank $r_{n}$, number of rows $m_{n}$ and number of columns $n$ all tend to $\infty$ proportionally to each other ($r_{n}/m_{n}\rightarrow \rho$, $m_{n}/n\rightarrow \beta$), we evaluate the asymptotic minimax MSE $ \mathcal{M} (\rho,\beta)=\lim_{m_{n},n\rightarrow \infty}\inf_{\lambda}\sup_{\operatorname{rank}(X)\leq r_{n}}\operatorname{MSE}(X_{0},\hat{X}_{\lambda})$. Our formulas involve incomplete moments of the quarter- and semi-circle laws ($\beta=1$, square case) and the Marčenko–Pastur law ($\beta<1$, nonsquare case). For finite $m$ and $n$, we show that MSE increases as the nonzero singular values of $X_{0}$ grow larger. As a result, the finite-$n$ worst-case MSE, a quantity which can be evaluated numerically, is achieved when the signal $X_{0}$ is “infinitely strong.”

The nuclear norm penalization problem is solved by applying soft thresholding to the singular values of $Y$. We also derive the minimax threshold, namely the value $\lambda^{*}(\rho)$, which is the optimal place to threshold the singular values.

All these results are obtained for general (nonsquare, nonsymmetric) real matrices. Comparable results are obtained for square symmetric nonnegative-definite matrices.


Download Citation

David Donoho. Matan Gavish. "Minimax risk of matrix denoising by singular value thresholding." Ann. Statist. 42 (6) 2413 - 2440, December 2014.


Published: December 2014
First available in Project Euclid: 20 October 2014

zbMATH: 1310.62014
MathSciNet: MR3269984
Digital Object Identifier: 10.1214/14-AOS1257

Primary: 62C20 , 62H25
Secondary: 90C22 , 90C25

Keywords: matrix completion from Gaussian measurements , Matrix denoising , monotonicity of power functions of multivariate tests , nuclear norm minimization , optimal threshold , phase transition , singular value thresholding , Stein unbiased risk estimate

Rights: Copyright © 2014 Institute of Mathematical Statistics

Vol.42 • No. 6 • December 2014
Back to Top