On the non-asymptotic concentration of heteroskedastic Wishart-type matrix

Abstract

This paper focuses on the non-asymptotic concentration of the heteroskedastic Wishart-type matrices. Suppose Z is a $p_1$-by-$p_2$random matrix and $Z_{ij}\sim N(0,{\mathit{\sigma }}_{ij}^2)$ independently, we prove the expected spectral norm of Wishart matrix deviations (i.e., $\mathbb{E}‖Z{Z}^{\top }-\mathbb{E}Z{Z}^{\top }‖$) is upper bounded by

$$(1+\mathit{ϵ})\left\{2{\mathit{\sigma }}_C{\mathit{\sigma }}_R+{\mathit{\sigma }}_C^2+C{\mathit{\sigma }}_R{\mathit{\sigma }}_{\ast }\sqrt{log(p_1\wedge p_2)}+C{\mathit{\sigma }}_{\ast }^{2}log(p_1\wedge p_2)\right\},$$

where ${\mathit{\sigma }}_C^2:={max}_j{\sum }_{i=1}^{p_1}{\mathit{\sigma }}_{ij}^2$, ${\mathit{\sigma }}_R^2:={max}_i{\sum }_{j=1}^{p_2}{\mathit{\sigma }}_{ij}^2$ and ${\mathit{\sigma }}_{\ast }^2:={max}_{i,j}{\mathit{\sigma }}_{ij}^2$. A minimax lower bound is developed that matches this upper bound. Then, we derive the concentration inequalities, moments, and tail bounds for the heteroskedastic Wishart-type matrix under more general distributions, such as sub-Gaussian and heavy-tailed distributions. Next, we consider the cases where Z has homoskedastic columns or rows (i.e., ${\mathit{\sigma }}_{ij}\approx {\mathit{\sigma }}_i$ or ${\mathit{\sigma }}_{ij}\approx {\mathit{\sigma }}_j$) and derive the rate-optimal Wishart-type concentration bounds. Finally, we apply the developed tools to identify the sharp signal-to-noise ratio threshold for consistent clustering in the heteroskedastic clustering problem.