The central subspace of a pair of random variables $(y,\boldsymbol{x})\in \mathbb{R}^{p+1}$ is the minimal subspace $\mathcal{S}$ such that $y\perp\!\!\!\!\!\perp \boldsymbol{x}|P_{\mathcal{S}}\boldsymbol{x}$. In this paper, we consider the minimax rate of estimating the central space under the multiple index model $y=f(\boldsymbol{\beta }_{1}^{\tau }\boldsymbol{x},\boldsymbol{\beta }_{2}^{\tau }\boldsymbol{x},\ldots,\boldsymbol{\beta }_{d}^{\tau }\boldsymbol{x},\epsilon )$ with at most $s$ active predictors, where $\boldsymbol{x}\sim N(0,\boldsymbol{\Sigma })$ for some class of $\boldsymbol{\Sigma }$. We first introduce a large class of models depending on the smallest nonzero eigenvalue $\lambda $ of $\operatorname{var}(\mathbb{E}[\boldsymbol{x}|y])$, over which we show that an aggregated estimator based on the SIR procedure converges at rate $d\wedge ((sd+s\log (ep/s))/(n\lambda ))$. We then show that this rate is optimal in two scenarios, the single index models and the multiple index models with fixed central dimension $d$ and fixed $\lambda $. By assuming a technical conjecture, we can show that this rate is also optimal for multiple index models with bounded dimension of the central space.
Ann. Statist.
49(1):
1-20
(February 2021).
DOI: 10.1214/19-AOS1813
Abramovich, F., Benjamini, Y., Donoho, D. L. and Johnstone, I. M. (2006). Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 34 584–653. MR2281879 1092.62005 10.1214/009053606000000074 euclid.aos/1151418235Abramovich, F., Benjamini, Y., Donoho, D. L. and Johnstone, I. M. (2006). Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 34 584–653. MR2281879 1092.62005 10.1214/009053606000000074 euclid.aos/1151418235
Amini, A. A. and Wainwright, M. J. (2008). High-dimensional analysis of semidefinite relaxations for sparse principal components. In Information Theory, 2008. ISIT 2008. IEEE International Symposium on 2454–2458. IEEE. MR2541450 10.1214/08-AOS664 euclid.aos/1247836672Amini, A. A. and Wainwright, M. J. (2008). High-dimensional analysis of semidefinite relaxations for sparse principal components. In Information Theory, 2008. ISIT 2008. IEEE International Symposium on 2454–2458. IEEE. MR2541450 10.1214/08-AOS664 euclid.aos/1247836672
Berthet, Q. and Rigollet, P. (2013). Computational lower bounds for sparse pca. Preprint. Available at arXiv:1304.0828. 1304.0828 1277.62155 euclid.aos/1378386239Berthet, Q. and Rigollet, P. (2013). Computational lower bounds for sparse pca. Preprint. Available at arXiv:1304.0828. 1304.0828 1277.62155 euclid.aos/1378386239
Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705–1732. 1173.62022 10.1214/08-AOS620 euclid.aos/1245332830Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705–1732. 1173.62022 10.1214/08-AOS620 euclid.aos/1245332830
Birnbaum, A., Johnstone, I. M., Nadler, B. and Paul, D. (2013). Minimax bounds for sparse PCA with noisy high-dimensional data. Ann. Statist. 41 1055–1084. 1292.62071 10.1214/12-AOS1014 euclid.aos/1371150893Birnbaum, A., Johnstone, I. M., Nadler, B. and Paul, D. (2013). Minimax bounds for sparse PCA with noisy high-dimensional data. Ann. Statist. 41 1055–1084. 1292.62071 10.1214/12-AOS1014 euclid.aos/1371150893
Cai, T. T., Ma, Z. and Wu, Y. (2013). Sparse PCA: Optimal rates and adaptive estimation. Ann. Statist. 41 3074–3110. 1288.62099 euclid.aos/1388545679Cai, T. T., Ma, Z. and Wu, Y. (2013). Sparse PCA: Optimal rates and adaptive estimation. Ann. Statist. 41 3074–3110. 1288.62099 euclid.aos/1388545679
Cai, T. T. and Zhou, H. H. (2012). Optimal rates of convergence for sparse covariance matrix estimation. Ann. Statist. 40 2389–2420. 1373.62247 10.1214/12-AOS998 euclid.aos/1359987525Cai, T. T. and Zhou, H. H. (2012). Optimal rates of convergence for sparse covariance matrix estimation. Ann. Statist. 40 2389–2420. 1373.62247 10.1214/12-AOS998 euclid.aos/1359987525
Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when $p$ is much larger than $n$. Ann. Statist. 35 2313–2351. 1139.62019 10.1214/009053606000001523 euclid.aos/1201012958Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when $p$ is much larger than $n$. Ann. Statist. 35 2313–2351. 1139.62019 10.1214/009053606000001523 euclid.aos/1201012958
Chen, C.-H. and Li, K.-C. (1998). Can SIR be as popular as multiple linear regression? Statist. Sinica 8 289–316. 0897.62069Chen, C.-H. and Li, K.-C. (1998). Can SIR be as popular as multiple linear regression? Statist. Sinica 8 289–316. 0897.62069
Cook, R. D., Forzani, L. and Rothman, A. J. (2012). Estimating sufficient reductions of the predictors in abundant high-dimensional regressions. Ann. Statist. 40 353–384. 1246.62150 10.1214/11-AOS962 euclid.aos/1333567193Cook, R. D., Forzani, L. and Rothman, A. J. (2012). Estimating sufficient reductions of the predictors in abundant high-dimensional regressions. Ann. Statist. 40 353–384. 1246.62150 10.1214/11-AOS962 euclid.aos/1333567193
Dennis Cook, R. (1998). Regression Graphics: Ideas for Studying Regressions Through Graphics. Wiley Series in Probability and Statistics: Probability and Statistics. Wiley, New York. 0903.62001Dennis Cook, R. (1998). Regression Graphics: Ideas for Studying Regressions Through Graphics. Wiley Series in Probability and Statistics: Probability and Statistics. Wiley, New York. 0903.62001
Dennis Cook, R. (2000). Save: A method for dimension reduction and graphics in regression. Comm. Statist. Theory Methods 29 2109–2121. 1061.62503 10.1080/03610920008832598Dennis Cook, R. (2000). Save: A method for dimension reduction and graphics in regression. Comm. Statist. Theory Methods 29 2109–2121. 1061.62503 10.1080/03610920008832598
Dennis Cook, R. and Weisberg, S. (1991). Comment. J. Amer. Statist. Assoc. 86 328–332. 1353.62037Dennis Cook, R. and Weisberg, S. (1991). Comment. J. Amer. Statist. Assoc. 86 328–332. 1353.62037
Duan, N. and Li, K.-C. (1991). Slicing regression: A link-free regression method. Ann. Statist. 19 505–530. 0738.62070 10.1214/aos/1176348109 euclid.aos/1176348109Duan, N. and Li, K.-C. (1991). Slicing regression: A link-free regression method. Ann. Statist. 19 505–530. 0738.62070 10.1214/aos/1176348109 euclid.aos/1176348109
Hsing, T. and Carroll, R. J. (1992). An asymptotic theory for sliced inverse regression. Ann. Statist. 20 1040–1061. 0821.62019 10.1214/aos/1176348669 euclid.aos/1176348669Hsing, T. and Carroll, R. J. (1992). An asymptotic theory for sliced inverse regression. Ann. Statist. 20 1040–1061. 0821.62019 10.1214/aos/1176348669 euclid.aos/1176348669
Jung, S. and Marron, J. S. (2009). PCA consistency in high dimension, low sample size context. Ann. Statist. 37 4104–4130. 1191.62108 10.1214/09-AOS709 euclid.aos/1256303538Jung, S. and Marron, J. S. (2009). PCA consistency in high dimension, low sample size context. Ann. Statist. 37 4104–4130. 1191.62108 10.1214/09-AOS709 euclid.aos/1256303538
Li, K.-C. (1991). Sliced inverse regression for dimension reduction. J. Amer. Statist. Assoc. 86 316–342. 0742.62044 10.1080/01621459.1991.10475035Li, K.-C. (1991). Sliced inverse regression for dimension reduction. J. Amer. Statist. Assoc. 86 316–342. 0742.62044 10.1080/01621459.1991.10475035
Li, L. (2007). Sparse sufficient dimension reduction. Biometrika 94 603–613. 1135.62062 10.1093/biomet/asm044Li, L. (2007). Sparse sufficient dimension reduction. Biometrika 94 603–613. 1135.62062 10.1093/biomet/asm044
Li, B. and Wang, S. (2007). On directional regression for dimension reduction. J. Amer. Statist. Assoc. 102 997–1008. 05564427 10.1198/016214507000000536Li, B. and Wang, S. (2007). On directional regression for dimension reduction. J. Amer. Statist. Assoc. 102 997–1008. 05564427 10.1198/016214507000000536
Lin, Q., Zhao, Z. and Liu, J. S. (2018). On consistency and sparsity for sliced inverse regression in high dimensions. Ann. Statist. 46 580–610. 1395.62196 10.1214/17-AOS1561 euclid.aos/1522742430Lin, Q., Zhao, Z. and Liu, J. S. (2018). On consistency and sparsity for sliced inverse regression in high dimensions. Ann. Statist. 46 580–610. 1395.62196 10.1214/17-AOS1561 euclid.aos/1522742430
Lin, Q., Zhao, Z. and Liu, J. S. (2019). Sparse sliced inverse regression via Lasso. J. Amer. Statist. Assoc. 114 1726–1739. 1428.62320 10.1080/01621459.2018.1520115Lin, Q., Zhao, Z. and Liu, J. S. (2019). Sparse sliced inverse regression via Lasso. J. Amer. Statist. Assoc. 114 1726–1739. 1428.62320 10.1080/01621459.2018.1520115
Lin, Q., Li, X., Huang, D. and Liu, J. S. (2021). Supplement to “On the optimality of sliced inverse regression in high dimensions.” https://doi.org/10.1214/19-AOS1813SUPPLin, Q., Li, X., Huang, D. and Liu, J. S. (2021). Supplement to “On the optimality of sliced inverse regression in high dimensions.” https://doi.org/10.1214/19-AOS1813SUPP
Neykov, M., Lin, Q. and Liu, J. S. (2016). Signed support recovery for single index models in high-dimensions. Ann. Math. Sci. Appl. 1 379–426. MR3876481 1392.62175 10.4310/AMSA.2016.v1.n2.a5Neykov, M., Lin, Q. and Liu, J. S. (2016). Signed support recovery for single index models in high-dimensions. Ann. Math. Sci. Appl. 1 379–426. MR3876481 1392.62175 10.4310/AMSA.2016.v1.n2.a5
Raskutti, G., Wainwright, M. J. and Yu, B. (2011). Minimax rates of estimation for high-dimensional linear regression over $\ell_{q}$-balls. IEEE Trans. Inf. Theory 57 6976–6994. 1365.62276 10.1109/TIT.2011.2165799Raskutti, G., Wainwright, M. J. and Yu, B. (2011). Minimax rates of estimation for high-dimensional linear regression over $\ell_{q}$-balls. IEEE Trans. Inf. Theory 57 6976–6994. 1365.62276 10.1109/TIT.2011.2165799
Schott, J. R. (1994). Determining the dimensionality in sliced inverse regression. J. Amer. Statist. Assoc. 89 141–148. 0791.62069 10.1080/01621459.1994.10476455Schott, J. R. (1994). Determining the dimensionality in sliced inverse regression. J. Amer. Statist. Assoc. 89 141–148. 0791.62069 10.1080/01621459.1994.10476455
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288. 0850.62538 10.1111/j.2517-6161.1996.tb02080.xTibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288. 0850.62538 10.1111/j.2517-6161.1996.tb02080.x
Vu, V. Q. and Lei, J. (2012). Minimax rates of estimation for sparse pca in high dimensions. Preprint. Available at arXiv:1202.0786. 1202.0786Vu, V. Q. and Lei, J. (2012). Minimax rates of estimation for sparse pca in high dimensions. Preprint. Available at arXiv:1202.0786. 1202.0786
Yang, S. S. (1977). General distribution theory of the concomitants of order statistics. Ann. Statist. 5 996–1002. 0367.62017 10.1214/aos/1176343954 euclid.aos/1176343954Yang, S. S. (1977). General distribution theory of the concomitants of order statistics. Ann. Statist. 5 996–1002. 0367.62017 10.1214/aos/1176343954 euclid.aos/1176343954
Zhu, L., Miao, B. and Peng, H. (2006). On sliced inverse regression with high-dimensional covariates. J. Amer. Statist. Assoc. 101 630–643. 1119.62331 10.1198/016214505000001285Zhu, L., Miao, B. and Peng, H. (2006). On sliced inverse regression with high-dimensional covariates. J. Amer. Statist. Assoc. 101 630–643. 1119.62331 10.1198/016214505000001285
Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 301–320. 1069.62054 10.1111/j.1467-9868.2005.00503.xZou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 301–320. 1069.62054 10.1111/j.1467-9868.2005.00503.x