Geometric sharp large deviations for random projections of ℓpn spheres and balls

Abstract

Accurate estimation of tail probabilities of projections of high-dimensional probability measures is of relevance in high-dimensional statistics and asymptotic geometric analysis. Whereas large deviation principles identify the asymptotic exponential decay rate of probabilities, sharp large deviation estimates also provide the “prefactor” in front of the exponentially decaying term. For fixed $p\in (1,\mathrm{\infty })$, consider independent sequences ${(X^{(n,p)})}_{n\in \mathbb{N}}$ and ${({\mathrm{\Theta }}^n)}_{n\in \mathbb{N}}$ of random vectors with ${\mathrm{\Theta }}^n$ distributed according to the normalized cone measure on the unit ${\ell }_2^n$ sphere, and $X^{(n,p)}$ distributed according to the normalized cone measure on the unit ${\ell }_p^n$ sphere. For almost every realization ${({\mathit{\theta }}^n)}_{n\in \mathbb{N}}$ of ${({\mathrm{\Theta }}^n)}_{n\in \mathbb{N}}$, (quenched) sharp large deviation estimates are established for suitably normalized (scalar) projections of $X^{(n,p)}$ onto ${\mathit{\theta }}^n$, that are asymptotically exact (as the dimension n tends to infinity). Furthermore, the case when ${(X^{(n,p)})}_{n\in \mathbb{N}}$ is replaced with ${(X^{(n,p)})}_{n\in \mathbb{N}}$, where $X^{(n,p)}$ is distributed according to the uniform (or normalized volume) measure on the unit ${\ell }_p^n$ ball, is also considered. In both cases, in contrast to the (quenched) large deviation rate function, the prefactor exhibits a dependence on the projection directions ${({\mathit{\theta }}^n)}_{n\in \mathbb{N}}$ that encodes additional geometric information that enables one to distinguish between projections of balls and spheres. Moreover, comparison with numerical estimates obtained by direct computation and importance sampling shows that the obtained analytical expressions for tail probabilities provide good approximations even for moderate values of n. The results on the one hand provide more accurate quantitative estimates of tail probabilities of random projections of ${\ell }_p^n$ spheres than logarithmic asymptotics, and on the other hand, generalize classical sharp large deviation estimates in the spirit of Bahadur and Ranga Rao to a geometric setting. The proofs combine Fourier analytic and probabilistic techniques. Along the way, several results of independent interest are obtained including a simpler representation for the quenched large deviation rate function that shows that it is strictly convex, a central limit theorem for random projections under a certain family of tilted measures, and multi-dimensional generalized Laplace asymptotics.