Open Access
June 2015 Computational barriers in minimax submatrix detection
Zongming Ma, Yihong Wu
Ann. Statist. 43(3): 1089-1116 (June 2015). DOI: 10.1214/14-AOS1300

Abstract

This paper studies the minimax detection of a small submatrix of elevated mean in a large matrix contaminated by additive Gaussian noise. To investigate the tradeoff between statistical performance and computational cost from a complexity-theoretic perspective, we consider a sequence of discretized models which are asymptotically equivalent to the Gaussian model. Under the hypothesis that the planted clique detection problem cannot be solved in randomized polynomial time when the clique size is of smaller order than the square root of the graph size, the following phase transition phenomenon is established: when the size of the large matrix $p\to\infty$, if the submatrix size $k=\Theta(p^{\alpha})$ for any $\alpha\in(0,{2}/{3})$, computational complexity constraints can incur a severe penalty on the statistical performance in the sense that any randomized polynomial-time test is minimax suboptimal by a polynomial factor in $p$; if $k=\Theta(p^{\alpha})$ for any $\alpha\in({2}/{3},1)$, minimax optimal detection can be attained within constant factors in linear time. Using Schatten norm loss as a representative example, we show that the hardness of attaining the minimax estimation rate can crucially depend on the loss function. Implications on the hardness of support recovery are also obtained.

Citation

Download Citation

Zongming Ma. Yihong Wu. "Computational barriers in minimax submatrix detection." Ann. Statist. 43 (3) 1089 - 1116, June 2015. https://doi.org/10.1214/14-AOS1300

Information

Received: 1 August 2014; Revised: 1 December 2014; Published: June 2015
First available in Project Euclid: 15 May 2015

zbMATH: 1328.62354
MathSciNet: MR3346698
Digital Object Identifier: 10.1214/14-AOS1300

Subjects:
Primary: 62H15
Secondary: 62C20

Keywords: ‎asymptotic ‎equivalence , computational complexity , High-dimensional statistics , Minimax rate , planted clique , submatrix detection

Rights: Copyright © 2015 Institute of Mathematical Statistics

Vol.43 • No. 3 • June 2015
Back to Top