Inference in latent factor regression with clusterable features

Abstract

Regression models, in which the observed features $\mathit{X}\in {\mathbb{R}}^{\mathit{p}}$ and the response $\mathit{Y}\in \mathbb{R}$ depend, jointly, on a lower dimensional, unobserved, latent vector $\mathit{Z}\in {\mathbb{R}}^{\mathit{K}}$, with $\mathit{K}\ll \mathit{p}$, are popular in a large array of applications, and mainly used for predicting a response from correlated features. In contrast, methodology and theory for inference on the regression coefficient $\mathit{\beta }\in {\mathbb{R}}^{\mathit{K}}$ relating Y to Z are scarce, since typically the un-observable factor Z is hard to interpret. Furthermore, the determination of the asymptotic variance of an estimator of β is a long-standing problem, with solutions known only in a few particular cases.

To address some of these outstanding questions, we develop inferential tools for β in a class of factor regression models in which the observed features are signed mixtures of the latent factors. The model specifications are both practically desirable, in a large array of applications, render interpretability to the components of Z, and are sufficient for parameter identifiability.

Without assuming that the number of latent factors K or the structure of the mixture is known in advance, we construct computationally efficient estimators of β, along with estimators of other important model parameters. We benchmark the rate of convergence of β by first establishing its ${\ell }_2$-norm minimax lower bound, and show that our proposed estimator $\hat{\mathit{\beta }}$ is minimax-rate adaptive. Our main contribution is the provision of a unified analysis of the component-wise Gaussian asymptotic distribution of $\hat{\mathit{\beta }}$ and, especially, the derivation of a closed form expression of its asymptotic variance, together with consistent variance estimators. The resulting inferential tools can be used when both K and p are independent of the sample size n, and also when both, or either, p and K vary with n, while allowing for $\mathit{p}>\mathit{n}$. This complements the only asymptotic normality results obtained for a particular case of the model under consideration, in the regime $\mathit{K}=\mathit{O}(1)$ and $\mathit{p}\to \infty$, but without a variance estimate.

As an application, we provide, within our model specifications, a statistical platform for inference in regression on latent cluster centers, thereby increasing the scope of our theoretical results.

We benchmark the newly developed methodology on a recently collected data set for the study of the effectiveness of a new SIV vaccine. Our analysis enables the determination of the top latent antibody-centric mechanisms associated with the vaccine response.