Minimax risk of matrix denoising by singular value thresholding

Abstract

An unknown $m$ by $n$ matrix $X_{0}$ is to be estimated from noisy measurements $Y=X_{0}+Z$, where the noise matrix $Z$ has i.i.d. Gaussian entries. A popular matrix denoising scheme solves the nuclear norm penalization problem $\operatorname{min}_{X}\|Y-X\|_{F}^{2}/2+\lambda\|X\|_{*}$, where $\|X\|_{*}$ denotes the nuclear norm (sum of singular values). This is the analog, for matrices, of $\ell_{1}$ penalization in the vector case. It has been empirically observed that if $X_{0}$ has low rank, it may be recovered quite accurately from the noisy measurement $Y$.

In a proportional growth framework where the rank $r_{n}$, number of rows $m_{n}$ and number of columns $n$ all tend to $\infty$ proportionally to each other ($r_{n}/m_{n}\rightarrow \rho$, $m_{n}/n\rightarrow \beta$), we evaluate the asymptotic minimax MSE $ \mathcal{M} (\rho,\beta)=\lim_{m_{n},n\rightarrow \infty}\inf_{\lambda}\sup_{\operatorname{rank}(X)\leq r_{n}}\operatorname{MSE}(X_{0},\hat{X}_{\lambda})$. Our formulas involve incomplete moments of the quarter- and semi-circle laws ($\beta=1$, square case) and the Marčenko–Pastur law ($\beta<1$, nonsquare case). For finite $m$ and $n$, we show that MSE increases as the nonzero singular values of $X_{0}$ grow larger. As a result, the finite-$n$ worst-case MSE, a quantity which can be evaluated numerically, is achieved when the signal $X_{0}$ is “infinitely strong.”

The nuclear norm penalization problem is solved by applying soft thresholding to the singular values of $Y$. We also derive the minimax threshold, namely the value $\lambda^{*}(\rho)$, which is the optimal place to threshold the singular values.

All these results are obtained for general (nonsquare, nonsymmetric) real matrices. Comparable results are obtained for square symmetric nonnegative-definite matrices.