Open Access
2022 Kronecker-structured covariance models for multiway data
Yu Wang, Zeyu Sun, Dogyoon Song, Alfred Hero
Author Affiliations +
Statist. Surv. 16: 238-270 (2022). DOI: 10.1214/22-SS139
Abstract

Many applications produce multiway data of exceedingly high dimension. Modeling such multi-way data is important in multichannel signal and video processing where sensors produce multi-indexed data, e.g. over spatial, frequency, and temporal dimensions. We will address the challenges of covariance representation of multiway data and review some of the progress in statistical modeling of multiway covariance over the past two decades, focusing on tensor-valued covariance models and their inference. We will illustrate through a space weather application: predicting the evolution of solar active regions over time.

References

1.

Akdemir, D. and Gupta, A. K. (2011). Array Variate Random Variables with Multiway Kronecker Delta Covariance Matrix Structure. Journal of algebraic statistics 2.Akdemir, D. and Gupta, A. K. (2011). Array Variate Random Variables with Multiway Kronecker Delta Covariance Matrix Structure. Journal of algebraic statistics 2.

2.

Allen, G. I. and Tibshirani, R. (2010). Transposable regularized covariance models with an application to missing data imputation. The Annals of Applied Statistics 4 764.Allen, G. I. and Tibshirani, R. (2010). Transposable regularized covariance models with an application to missing data imputation. The Annals of Applied Statistics 4 764.

3.

Allen, G. I. and Tibshirani, R. (2012). Inference with transposable data: modelling the effects of row and column correlations. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 74 721–743.Allen, G. I. and Tibshirani, R. (2012). Inference with transposable data: modelling the effects of row and column correlations. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 74 721–743.

4.

Bai, L., Bi, Y., Yang, B., Hong, J.-C., Xu, Z., Shang, Z.-H., Liu, H., Ji, H.-S. and Ji, K.-F. (2021). Predicting the evolution of photospheric magnetic field in solar active regions using deep learning. Research in Astronomy and Astrophysics 21 113.Bai, L., Bi, Y., Yang, B., Hong, J.-C., Xu, Z., Shang, Z.-H., Liu, H., Ji, H.-S. and Ji, K.-F. (2021). Predicting the evolution of photospheric magnetic field in solar active regions using deep learning. Research in Astronomy and Astrophysics 21 113.

5.

Banerjee, O., El Ghaoui, L. and d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. The Journal of Machine Learning Research 9 485–516.  MR2417243Banerjee, O., El Ghaoui, L. and d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. The Journal of Machine Learning Research 9 485–516.  MR2417243

6.

Besag, J. (1977). Efficiency of pseudolikelihood estimation for simple Gaussian fields. Biometrika 616–618.Besag, J. (1977). Efficiency of pseudolikelihood estimation for simple Gaussian fields. Biometrika 616–618.

7.

Bijma, F., De Munck, J. C. and Heethaar, R. M. (2005). The spatiotemporal MEG covariance matrix modeled as a sum of Kronecker products. NeuroImage 27 402–415.Bijma, F., De Munck, J. C. and Heethaar, R. M. (2005). The spatiotemporal MEG covariance matrix modeled as a sum of Kronecker products. NeuroImage 27 402–415.

8.

Box, G. E., Jenkins, G. M., Reinsel, G. C. and Ljung, G. M. (2015). Time series analysis: forecasting and control. John Wiley & Sons.Box, G. E., Jenkins, G. M., Reinsel, G. C. and Ljung, G. M. (2015). Time series analysis: forecasting and control. John Wiley & Sons.

9.

Candès, E. J., Li, X., Ma, Y. and Wright, J. (2011). Robust principal component analysis? Journal of the ACM (JACM) 58 1–37.  MR2811000Candès, E. J., Li, X., Ma, Y. and Wright, J. (2011). Robust principal component analysis? Journal of the ACM (JACM) 58 1–37.  MR2811000

10.

Chandrasekaran, V. and Jordan, M. I. (2013). Computational and statistical tradeoffs via convex relaxation. Proceedings of the National Academy of Sciences 110 E1181–E1190.Chandrasekaran, V. and Jordan, M. I. (2013). Computational and statistical tradeoffs via convex relaxation. Proceedings of the National Academy of Sciences 110 E1181–E1190.

11.

Chandrasekaran, V., Sanghavi, S., Parrilo, P. A. and Willsky, A. S. (2011). Rank-sparsity incoherence for matrix decomposition. SIAM Journal on Optimization 21 572–596.Chandrasekaran, V., Sanghavi, S., Parrilo, P. A. and Willsky, A. S. (2011). Rank-sparsity incoherence for matrix decomposition. SIAM Journal on Optimization 21 572–596.

12.

Chandrasekhar, S. (1943). Stochastic problems in physics and astronomy. Reviews of modern physics 15 1.Chandrasekhar, S. (1943). Stochastic problems in physics and astronomy. Reviews of modern physics 15 1.

13.

Chen, Y., Wiesel, A. and Hero, A. O. (2011). Robust shrinkage estimation of high-dimensional covariance matrices. IEEE Transactions on Signal Processing 59 4097–4107.Chen, Y., Wiesel, A. and Hero, A. O. (2011). Robust shrinkage estimation of high-dimensional covariance matrices. IEEE Transactions on Signal Processing 59 4097–4107.

14.

Chen, Y., Manchester, W. B., Hero, A. O., Toth, G., DuFumier, B., Zhou, T., Wang, X., Zhu, H., Sun, Z. and Gombosi, T. I. (2019). Identifying solar flare precursors using time series of SDO/HMI images and SHARP parameters. Space Weather 17 1404–1426.Chen, Y., Manchester, W. B., Hero, A. O., Toth, G., DuFumier, B., Zhou, T., Wang, X., Zhu, H., Sun, Z. and Gombosi, T. I. (2019). Identifying solar flare precursors using time series of SDO/HMI images and SHARP parameters. Space Weather 17 1404–1426.

15.

Chu, M. T. and Lin, M. M. (2021). Nonlinear Power-Like and SVD-Like Iterative Schemes with Applications to Entangled Bipartite Rank-1 Approximation. SIAM Journal on Scientific Computing 0 S448–S474.  MR4331954Chu, M. T. and Lin, M. M. (2021). Nonlinear Power-Like and SVD-Like Iterative Schemes with Applications to Entangled Bipartite Rank-1 Approximation. SIAM Journal on Scientific Computing 0 S448–S474.  MR4331954

16.

Cressie, N. (2015). Statistics for spatial data. John Wiley & Sons.Cressie, N. (2015). Statistics for spatial data. John Wiley & Sons.

17.

Dai, B., Wang, Y., Aston, J., Hua, G. and Wipf, D. (2018). Connections with robust PCA and the role of emergent sparsity in variational autoencoder models. The Journal of Machine Learning Research 19 1573–1614.Dai, B., Wang, Y., Aston, J., Hua, G. and Wipf, D. (2018). Connections with robust PCA and the role of emergent sparsity in variational autoencoder models. The Journal of Machine Learning Research 19 1573–1614.

18.

Dantas, C. F., Cohen, J. E. and Gribonval, R. (2019). Learning tensor-structured dictionaries with application to hyperspectral image denoising. In 2019 27th European Signal Processing Conference (EUSIPCO) 1–5. IEEE.Dantas, C. F., Cohen, J. E. and Gribonval, R. (2019). Learning tensor-structured dictionaries with application to hyperspectral image denoising. In 2019 27th European Signal Processing Conference (EUSIPCO) 1–5. IEEE.

19.

Dawid, A. P. (1981). Some matrix-variate distribution theory: notational considerations and a Bayesian application. Biometrika 68 265–274.Dawid, A. P. (1981). Some matrix-variate distribution theory: notational considerations and a Bayesian application. Biometrika 68 265–274.

20.

Deng, Y., Tang, X. and Qu, A. (2021). Correlation Tensor Decomposition and Its Application in Spatial Imaging Data. Journal of the American Statistical Association just-accepted 1–35.Deng, Y., Tang, X. and Qu, A. (2021). Correlation Tensor Decomposition and Its Application in Spatial Imaging Data. Journal of the American Statistical Association just-accepted 1–35.

21.

Du, J., Yan, X., Liu, Z., Cui, L., Ding, P., Tan, X., Li, X., Zhou, H., Gu, Q. and Xu, J. (2017). cBinderDB: a covalent binding agent database. Bioinformatics 33 1258–1260.Du, J., Yan, X., Liu, Z., Cui, L., Ding, P., Tan, X., Li, X., Zhou, H., Gu, Q. and Xu, J. (2017). cBinderDB: a covalent binding agent database. Bioinformatics 33 1258–1260.

22.

Dutilleul, P. (1999). The MLE algorithm for the matrix normal distribution. Journal of statistical computation and simulation 64 105–123.Dutilleul, P. (1999). The MLE algorithm for the matrix normal distribution. Journal of statistical computation and simulation 64 105–123.

23.

Evensen, G. (1994). Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. Journal of Geophysical Research: Oceans 99 10143–10162.Evensen, G. (1994). Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. Journal of Geophysical Research: Oceans 99 10143–10162.

24.

Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432–441.Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432–441.

25.

Fu, Y. and Huang, T. S. (2008). Image classification using correlation tensor analysis. IEEE Transactions on Image Processing 17 226–234.  MR2446011Fu, Y. and Huang, T. S. (2008). Image classification using correlation tensor analysis. IEEE Transactions on Image Processing 17 226–234.  MR2446011

26.

Galecki, A. T. (1994). General class of covariance structures for two or more repeated factors in longitudinal data analysis. Communications in Statistics-Theory and Methods 23 3105–3119.Galecki, A. T. (1994). General class of covariance structures for two or more repeated factors in longitudinal data analysis. Communications in Statistics-Theory and Methods 23 3105–3119.

27.

Galvez, R., Fouhey, D. F., Jin, M., Szenicer, A., Muñoz-Jaramillo, A., Cheung, M. C., Wright, P. J., Bobra, M. G., Liu, Y., Mason, J. et al. (2019). A machine-learning data set prepared from the NASA solar dynamics observatory mission. The Astrophysical Journal Supplement Series 242 7.Galvez, R., Fouhey, D. F., Jin, M., Szenicer, A., Muñoz-Jaramillo, A., Cheung, M. C., Wright, P. J., Bobra, M. G., Liu, Y., Mason, J. et al. (2019). A machine-learning data set prepared from the NASA solar dynamics observatory mission. The Astrophysical Journal Supplement Series 242 7.

28.

Ge, R. and Ma, T. (2020). On the optimization landscape of tensor decompositions. Mathematical Programming 1–47.Ge, R. and Ma, T. (2020). On the optimization landscape of tensor decompositions. Mathematical Programming 1–47.

29.

Grasedyck, L. (2004). Existence and computation of low Kronecker-rank approximations for large linear systems of tensor product structure. Computing 72 247–265.Grasedyck, L. (2004). Existence and computation of low Kronecker-rank approximations for large linear systems of tensor product structure. Computing 72 247–265.

30.

Greenewald, K. H. and Hero, A. O. (2014). Kronecker PCA based spatio-temporal modeling of video for dismount classification. In Algorithms for Synthetic Aperture Radar Imagery XXI 9093 90930V. International Society for Optics and Photonics.Greenewald, K. H. and Hero, A. O. (2014). Kronecker PCA based spatio-temporal modeling of video for dismount classification. In Algorithms for Synthetic Aperture Radar Imagery XXI 9093 90930V. International Society for Optics and Photonics.

31.

Greenewald, K. and Hero, A. O. (2015). Robust kronecker product PCA for spatio-temporal covariance estimation. IEEE Transactions on Signal Processing 63 6368–6378.Greenewald, K. and Hero, A. O. (2015). Robust kronecker product PCA for spatio-temporal covariance estimation. IEEE Transactions on Signal Processing 63 6368–6378.

32.

Greenewald, K., Zelnio, E. and Hero, A. H. (2016). Robust SAR STAP via Kronecker decomposition. IEEE Transactions on Aerospace and Electronic Systems 52 2612–2625.Greenewald, K., Zelnio, E. and Hero, A. H. (2016). Robust SAR STAP via Kronecker decomposition. IEEE Transactions on Aerospace and Electronic Systems 52 2612–2625.

33.

Greenewald, K., Zhou, S. and Hero, A. (2019). Tensor graphical lasso (TeraLasso). Journal of the Royal Statistical Society: Series B (Statistical Methodology) 81 901–931.Greenewald, K., Zhou, S. and Hero, A. (2019). Tensor graphical lasso (TeraLasso). Journal of the Royal Statistical Society: Series B (Statistical Methodology) 81 901–931.

34.

Hao, B., Wang, B., Wang, P., Zhang, J., Yang, J. and Sun, W. W. (2021). Sparse tensor additive regression. Journal of Machine Learning Research 22 1–43.Hao, B., Wang, B., Wang, P., Zhang, J., Yang, J. and Sun, W. W. (2021). Sparse tensor additive regression. Journal of Machine Learning Research 22 1–43.

35.

Hillar, C. J. and Lim, L.-H. (2013). Most tensor problems are NP-hard. Journal of the ACM (JACM) 60 1–39.  MR3144915Hillar, C. J. and Lim, L.-H. (2013). Most tensor problems are NP-hard. Journal of the ACM (JACM) 60 1–39.  MR3144915

36.

Hoff, P. D. (2011). Separable covariance arrays via the Tucker product, with applications to multivariate relational data. Bayesian Analysis 6 179–196.Hoff, P. D. (2011). Separable covariance arrays via the Tucker product, with applications to multivariate relational data. Bayesian Analysis 6 179–196.

37.

Hopkins, S. B., Shi, J. and Steurer, D. (2015). Tensor principal component analysis via sum-of-square proofs. In Conference on Learning Theory 956–1006. PMLR.Hopkins, S. B., Shi, J. and Steurer, D. (2015). Tensor principal component analysis via sum-of-square proofs. In Conference on Learning Theory 956–1006. PMLR.

38.

Hori, T., Montcho, D., Agbangla, C., Ebana, K., Futakuchi, K. and Iwata, H. (2016). Multi-task Gaussian process for imputing missing data in multi-trait and multi-environment trials. Theoretical and Applied Genetics 129 2101–2115.Hori, T., Montcho, D., Agbangla, C., Ebana, K., Futakuchi, K. and Iwata, H. (2016). Multi-task Gaussian process for imputing missing data in multi-trait and multi-environment trials. Theoretical and Applied Genetics 129 2101–2115.

39.

Hou, E., Lawrence, E. and Hero, A. O. (2021). Penalized ensemble Kalman filters for high dimensional non-linear systems. PloS one 16 e0248046.Hou, E., Lawrence, E. and Hero, A. O. (2021). Penalized ensemble Kalman filters for high dimensional non-linear systems. PloS one 16 e0248046.

40.

Jiang, Y., Bigot, J. and Maabout, S. (2021). Online Graph Topology Learning from Matrix-valued Time Series. arXiv preprint arXiv: 2107.08020.Jiang, Y., Bigot, J. and Maabout, S. (2021). Online Graph Topology Learning from Matrix-valued Time Series. arXiv preprint arXiv: 2107.08020.

41.

Jiao, Z., Sun, H., Wang, X., Manchester, W., Gombosi, T., Hero, A. and Chen, Y. (2020). Solar flare intensity prediction with machine learning models. Space Weather 18 e2020SW002440.Jiao, Z., Sun, H., Wang, X., Manchester, W., Gombosi, T., Hero, A. and Chen, Y. (2020). Solar flare intensity prediction with machine learning models. Space Weather 18 e2020SW002440.

42.

Jolliffe, I. T. (1986). Principal components in regression analysis. In Principal component analysis 129–155. Springer.Jolliffe, I. T. (1986). Principal components in regression analysis. In Principal component analysis 129–155. Springer.

43.

Kalaitzis, A., Lafferty, J., Lawrence, N. D. and Zhou, S. (2013). The bigraphical lasso. In International Conference on Machine Learning 1229–1237. PMLR.Kalaitzis, A., Lafferty, J., Lawrence, N. D. and Zhou, S. (2013). The bigraphical lasso. In International Conference on Machine Learning 1229–1237. PMLR.

44.

Ke, Y., Minsker, S., Ren, Z., Sun, Q. and Zhou, W.-X. (2019). User-friendly covariance estimation for heavy-tailed distributions. Statistical Science 34 454–471.Ke, Y., Minsker, S., Ren, Z., Sun, Q. and Zhou, W.-X. (2019). User-friendly covariance estimation for heavy-tailed distributions. Statistical Science 34 454–471.

45.

Khare, K., Oh, S.-Y. and Rajaratnam, B. (2015). A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees. Journal of the Royal Statistical Society: Series B: Statistical Methodology 803–825.  MR3382598Khare, K., Oh, S.-Y. and Rajaratnam, B. (2015). A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees. Journal of the Royal Statistical Society: Series B: Statistical Methodology 803–825.  MR3382598

46.

Kolda, T. G. and Bader, B. W. (2009). Tensor decompositions and applications. SIAM review 51 455–500.Kolda, T. G. and Bader, B. W. (2009). Tensor decompositions and applications. SIAM review 51 455–500.

47.

Kressner, D. and Tobler, C. (2010). Krylov subspace methods for linear systems with tensor product structure. SIAM journal on matrix analysis and applications 31 1688–1714.Kressner, D. and Tobler, C. (2010). Krylov subspace methods for linear systems with tensor product structure. SIAM journal on matrix analysis and applications 31 1688–1714.

48.

Landsberg, J. (2011). Tensors: Geometry and Applications: Geometry and Applications 128. American Mathematical Soc.Landsberg, J. (2011). Tensors: Geometry and Applications: Geometry and Applications 128. American Mathematical Soc.

49.

Ledoit, O. and Wolf, M. (2020). The power of (non-) linear shrinking: A review and guide to covariance matrix estimation. Journal of Financial Econometrics.Ledoit, O. and Wolf, M. (2020). The power of (non-) linear shrinking: A review and guide to covariance matrix estimation. Journal of Financial Econometrics.

50.

Leka, K. and Barnes, G. (2003). Photospheric magnetic field properties of flaring versus flare-quiet active regions. II. Discriminant analysis. The Astrophysical Journal 595 1296.Leka, K. and Barnes, G. (2003). Photospheric magnetic field properties of flaring versus flare-quiet active regions. II. Discriminant analysis. The Astrophysical Journal 595 1296.

51.

Lemen, J. R., Akin, D. J., Boerner, P. F., Chou, C., Drake, J. F., Duncan, D. W., Edwards, C. G., Friedlaender, F. M., Heyman, G. F., Hurlburt, N. E. et al. (2011). The atmospheric imaging assembly (AIA) on the solar dynamics observatory (SDO). In The solar dynamics observatory 17–40. Springer.Lemen, J. R., Akin, D. J., Boerner, P. F., Chou, C., Drake, J. F., Duncan, D. W., Edwards, C. G., Friedlaender, F. M., Heyman, G. F., Hurlburt, N. E. et al. (2011). The atmospheric imaging assembly (AIA) on the solar dynamics observatory (SDO). In The solar dynamics observatory 17–40. Springer.

52.

Leng, C. and Pan, G. (2018). Covariance estimation via sparse Kronecker structures. Bernoulli 24 3833–3863.Leng, C. and Pan, G. (2018). Covariance estimation via sparse Kronecker structures. Bernoulli 24 3833–3863.

53.

Li, L. and Zhang, X. (2017). Parsimonious tensor response regression. Journal of the American Statistical Association 112 1131–1146.Li, L. and Zhang, X. (2017). Parsimonious tensor response regression. Journal of the American Statistical Association 112 1131–1146.

54.

Li, Z., Chao, Y., McWilliams, J. C. and Ide, K. (2008). A three-dimensional variational data assimilation scheme for the regional ocean modeling system. Journal of Atmospheric and Oceanic Technology 25 2074–2090.Li, Z., Chao, Y., McWilliams, J. C. and Ide, K. (2008). A three-dimensional variational data assimilation scheme for the regional ocean modeling system. Journal of Atmospheric and Oceanic Technology 25 2074–2090.

55.

Li, S., López-García, M., Lawrence, N. D. and Cutillo, L. (2022). Scalable Bigraphical Lasso: Two-way Sparse Network Inference for Count Data. arXiv preprint arXiv: 2203.07912.Li, S., López-García, M., Lawrence, N. D. and Cutillo, L. (2022). Scalable Bigraphical Lasso: Two-way Sparse Network Inference for Count Data. arXiv preprint arXiv: 2203.07912.

56.

Lindgren, F., Rue, H. and Lindström, J. (2011). An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 423–498.  MR2853727Lindgren, F., Rue, H. and Lindström, J. (2011). An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 423–498.  MR2853727

57.

Liu, G., Tan, X., Dang, C., Tan, S., Xing, S., Huang, N., Peng, K., Xie, C., Tang, X. and Zeng, J. (2019). Regional shape abnormalities in thalamus and verbal memory impairment after subcortical infarction. Neurorehabilitation and Neural Repair 33 476–485.Liu, G., Tan, X., Dang, C., Tan, S., Xing, S., Huang, N., Peng, K., Xie, C., Tang, X. and Zeng, J. (2019). Regional shape abnormalities in thalamus and verbal memory impairment after subcortical infarction. Neurorehabilitation and Neural Repair 33 476–485.

58.

Llosa-Vite, C. and Maitra, R. (2022). Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance. IEEE Transactions on Pattern Analysis and Machine Intelligence 1–1.  10.1109/TPAMI.2022.3164836Llosa-Vite, C. and Maitra, R. (2022). Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance. IEEE Transactions on Pattern Analysis and Machine Intelligence 1–1.  10.1109/TPAMI.2022.3164836

59.

Lyu, X., Sun, W. W., Wang, Z., Liu, H., Yang, J. and Cheng, G. (2019). Tensor graphical model: Non-convex optimization and statistical inference. IEEE transactions on pattern analysis and machine intelligence 42 2024–2037.Lyu, X., Sun, W. W., Wang, Z., Liu, H., Yang, J. and Cheng, G. (2019). Tensor graphical model: Non-convex optimization and statistical inference. IEEE transactions on pattern analysis and machine intelligence 42 2024–2037.

60.

Manceur, A. M. and Dutilleul, P. (2013). Maximum likelihood estimation for the tensor normal distribution: Algorithm, minimum sample size, and empirical bias and dispersion. Journal of Computational and Applied Mathematics 239 37–49.Manceur, A. M. and Dutilleul, P. (2013). Maximum likelihood estimation for the tensor normal distribution: Algorithm, minimum sample size, and empirical bias and dispersion. Journal of Computational and Applied Mathematics 239 37–49.

61.

Mardia, K. V. and Goodall, C. R. (1993). Spatial-temporal analysis of multivariate environmental monitoring data. Multivariate environmental statistics 6 347–385.  MR1268438Mardia, K. V. and Goodall, C. R. (1993). Spatial-temporal analysis of multivariate environmental monitoring data. Multivariate environmental statistics 6 347–385.  MR1268438

62.

Matthews, B. W. (1975). Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405 442–451.Matthews, B. W. (1975). Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure 405 442–451.

63.

Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The annals of statistics 34 1436–1462.  MR2278363Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The annals of statistics 34 1436–1462.  MR2278363

64.

Min, K., Mai, Q. and Zhang, X. (2022). Fast and separable estimation in high-dimensional tensor Gaussian graphical models. Journal of Computational and Graphical Statistics 31 294–300.  MR4387228Min, K., Mai, Q. and Zhang, X. (2022). Fast and separable estimation in high-dimensional tensor Gaussian graphical models. Journal of Computational and Graphical Statistics 31 294–300.  MR4387228

65.

Mo, S., Xia, J., Tan, X. and Raj, B. (2022). Point3D: tracking actions as moving points with 3D CNNs. arXiv preprint arXiv: 2203.10584.Mo, S., Xia, J., Tan, X. and Raj, B. (2022). Point3D: tracking actions as moving points with 3D CNNs. arXiv preprint arXiv: 2203.10584.

66.

Molstad, A. J. and Rothman, A. J. (2019). A penalized likelihood method for classification with matrix-valued predictors. Journal of Computational and Graphical Statistics 28 11–22.Molstad, A. J. and Rothman, A. J. (2019). A penalized likelihood method for classification with matrix-valued predictors. Journal of Computational and Graphical Statistics 28 11–22.

67.

Ohlson, M., Ahmad, M. R. and Von Rosen, D. (2013). The multilinear normal distribution: Introduction and some basic properties. Journal of Multivariate Analysis 113 37–47.Ohlson, M., Ahmad, M. R. and Von Rosen, D. (2013). The multilinear normal distribution: Introduction and some basic properties. Journal of Multivariate Analysis 113 37–47.

68.

Perkel, J. M. (2019). Julia: come for the syntax, stay for the speed. Nature 572 141–143.Perkel, J. M. (2019). Julia: come for the syntax, stay for the speed. Nature 572 141–143.

69.

Pourahmadi, M. (2011). Covariance estimation: The GLM and regularization perspectives. Statistical Science 26 369–387.Pourahmadi, M. (2011). Covariance estimation: The GLM and regularization perspectives. Statistical Science 26 369–387.

70.

Pouryazdian, S., Beheshti, S. and Krishnan, S. (2016). CANDECOMP/PARAFAC model order selection based on reconstruction error in the presence of kronecker structured colored noise. Digital Signal Processing 48 12–26.Pouryazdian, S., Beheshti, S. and Krishnan, S. (2016). CANDECOMP/PARAFAC model order selection based on reconstruction error in the presence of kronecker structured colored noise. Digital Signal Processing 48 12–26.

71.

Rabusseau, G. and Kadri, H. (2016). Low-rank regression with tensor responses. Advances in Neural Information Processing Systems 29 1867–1875.Rabusseau, G. and Kadri, H. (2016). Low-rank regression with tensor responses. Advances in Neural Information Processing Systems 29 1867–1875.

72.

Richard, E. and Montanari, A. (2014). A statistical model for tensor PCA. Advances in Neural Information Processing Systems 27 2897–2905.Richard, E. and Montanari, A. (2014). A statistical model for tensor PCA. Advances in Neural Information Processing Systems 27 2897–2905.

73.

Roberts, B. (2006). Slow magnetohydrodynamic waves in the solar atmosphere. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 364 447–460.Roberts, B. (2006). Slow magnetohydrodynamic waves in the solar atmosphere. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 364 447–460.

74.

Stocker, T. (2011). Introduction to climate modelling. Springer Science & Business Media.Stocker, T. (2011). Introduction to climate modelling. Springer Science & Business Media.

75.

Strobach, P. (1995). Low-rank detection of multichannel Gaussian signals using block matrix approximation. IEEE Transactions on Signal Processing 43 233–242.Strobach, P. (1995). Low-rank detection of multichannel Gaussian signals using block matrix approximation. IEEE Transactions on Signal Processing 43 233–242.

76.

Sun, W. W., Hao, B. and Li, L. Tensors in Modern Statistical Learning.Sun, W. W., Hao, B. and Li, L. Tensors in Modern Statistical Learning.

77.

Sun, Z., Bobra, M., Wang, X., Wang, Y., Sun, H., Gombosi, T., Chen, Y. and Hero, A. (2021). Predicting Solar Flares using CNN and LSTM on Two Solar Cycles of Active Region Data. Earth and Space Science Open Archive 32.  10.1002/essoar.10508256.1Sun, Z., Bobra, M., Wang, X., Wang, Y., Sun, H., Gombosi, T., Chen, Y. and Hero, A. (2021). Predicting Solar Flares using CNN and LSTM on Two Solar Cycles of Active Region Data. Earth and Space Science Open Archive 32.  10.1002/essoar.10508256.1

78.

Tan, X., Ross, C. A., Miller, M. and Tang, X. (2018). Changepoint analysis of putamen and thalamus subregions in premanifest huntington’s disease. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) 531–535. IEEE.Tan, X., Ross, C. A., Miller, M. and Tang, X. (2018). Changepoint analysis of putamen and thalamus subregions in premanifest huntington’s disease. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) 531–535. IEEE.

79.

Tan, X., Qi, Z., Seymour, C. W. and Tang, L. (2022a). RISE: Robust Individualized Decision Learning with Sensitive Variables. arXiv preprint arXiv: 2211.06569.Tan, X., Qi, Z., Seymour, C. W. and Tang, L. (2022a). RISE: Robust Individualized Decision Learning with Sensitive Variables. arXiv preprint arXiv: 2211.06569.

80.

Tan, X., Abberbock, J., Rastogi, P. and Tang, G. (2022b). Identifying Principal Stratum Causal Effects Conditional on a Post-treatment Intermediate Response. In Conference on Causal Learning and Reasoning 734–753. PMLR.Tan, X., Abberbock, J., Rastogi, P. and Tang, G. (2022b). Identifying Principal Stratum Causal Effects Conditional on a Post-treatment Intermediate Response. In Conference on Causal Learning and Reasoning 734–753. PMLR.

81.

Tan, X., Yang, S., Ye, W., Faries, D. E., Lipkovich, I. and Kadziola, Z. (2022c). When Doubly Robust Methods Meet Machine Learning for Estimating Treatment Effects from Real-World Data: A Comparative Study. arXiv preprint arXiv: 2204.10969.Tan, X., Yang, S., Ye, W., Faries, D. E., Lipkovich, I. and Kadziola, Z. (2022c). When Doubly Robust Methods Meet Machine Learning for Estimating Treatment Effects from Real-World Data: A Comparative Study. arXiv preprint arXiv: 2204.10969.

82.

Tan, X., Chang, C.-C. H., Zhou, L. and Tang, L. (2022d). A tree-based model averaging approach for personalized treatment effect estimation from heterogeneous data sources. In International Conference on Machine Learning 21013–21036. PMLR.Tan, X., Chang, C.-C. H., Zhou, L. and Tang, L. (2022d). A tree-based model averaging approach for personalized treatment effect estimation from heterogeneous data sources. In International Conference on Machine Learning 21013–21036. PMLR.

83.

Teng, S. L. and Huang, H. (2009). A statistical framework to infer functional gene relationships from biologically interrelated microarray experiments. Journal of the American Statistical Association 104 465–473.  MR2751431Teng, S. L. and Huang, H. (2009). A statistical framework to infer functional gene relationships from biologically interrelated microarray experiments. Journal of the American Statistical Association 104 465–473.  MR2751431

84.

Tsiligkaridis, T. and Hero, A. O. (2013). Covariance estimation in high dimensions via kronecker product expansions. IEEE Transactions on Signal Processing 61 5347–5360.Tsiligkaridis, T. and Hero, A. O. (2013). Covariance estimation in high dimensions via kronecker product expansions. IEEE Transactions on Signal Processing 61 5347–5360.

85.

Tsiligkaridis, T., Hero, A. O. and Zhou, S. (2013). On convergence of kronecker graphical lasso algorithms. IEEE transactions on signal processing 61 1743–1755.Tsiligkaridis, T., Hero, A. O. and Zhou, S. (2013). On convergence of kronecker graphical lasso algorithms. IEEE transactions on signal processing 61 1743–1755.

86.

van Driel-Gesztelyi, L. and Green, L. M. (2015). Evolution of active regions. Living Reviews in Solar Physics 12 1–98.van Driel-Gesztelyi, L. and Green, L. M. (2015). Evolution of active regions. Living Reviews in Solar Physics 12 1–98.

87.

Van Loan, C. F. and Pitsianis, N. (1993). Approximation with Kronecker products. In Linear algebra for large scale and real-time applications 293–314. Springer.Van Loan, C. F. and Pitsianis, N. (1993). Approximation with Kronecker products. In Linear algebra for large scale and real-time applications 293–314. Springer.

88.

Varin, C., Reid, N. and Firth, D. (2011). An overview of composite likelihood methods. Statistica Sinica 5–42.Varin, C., Reid, N. and Firth, D. (2011). An overview of composite likelihood methods. Statistica Sinica 5–42.

89.

Wang, Y. and Hero, A. (2021a). Multiway Ensemble Kalman Filter. arXiv preprint arXiv: 2112.04322.Wang, Y. and Hero, A. (2021a). Multiway Ensemble Kalman Filter. arXiv preprint arXiv: 2112.04322.

90.

Wang, Y. and Hero, A. (2021b). SG-PALM: a Fast Physically Interpretable Tensor Graphical Model. International Conference on Machine Learning (ICML), arXiv preprint arXiv: 2105.12271.Wang, Y. and Hero, A. (2021b). SG-PALM: a Fast Physically Interpretable Tensor Graphical Model. International Conference on Machine Learning (ICML), arXiv preprint arXiv: 2105.12271.

91.

Wang, Y., Jang, B. and Hero, A. (2020). The sylvester graphical lasso (syglasso). In International Conference on Artificial Intelligence and Statistics 1943–1953. PMLR.Wang, Y., Jang, B. and Hero, A. (2020). The sylvester graphical lasso (syglasso). In International Conference on Artificial Intelligence and Statistics 1943–1953. PMLR.

92.

Wang, Y., Sun, Z. and Hero, A. (2022). TensorGraphicalModels: A Julia toolbox for multiway covariance models and ensemble Kalman filter. Software Impacts 13 100308.  https://doi.org/10.1016/j.simpa.2022.100308Wang, Y., Sun, Z. and Hero, A. (2022). TensorGraphicalModels: A Julia toolbox for multiway covariance models and ensemble Kalman filter. Software Impacts 13 100308.  https://doi.org/10.1016/j.simpa.2022.100308

93.

Wang, X., Chen, Y., Toth, G., Manchester, W. B., Gombosi, T. I., Hero, A. O., Jiao, Z., Sun, H., Jin, M. and Liu, Y. (2020). Predicting solar flares with machine learning: investigating solar cycle dependence. The Astrophysical Journal 895 3.Wang, X., Chen, Y., Toth, G., Manchester, W. B., Gombosi, T. I., Hero, A. O., Jiao, Z., Sun, H., Jin, M. and Liu, Y. (2020). Predicting solar flares with machine learning: investigating solar cycle dependence. The Astrophysical Journal 895 3.

94.

Wei, X. and Minsker, S. (2017). Estimation of the covariance structure of heavy-tailed distributions. In Advances in Neural Information Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan and R. Garnett, eds.) 30. Curran Associates, Inc.Wei, X. and Minsker, S. (2017). Estimation of the covariance structure of heavy-tailed distributions. In Advances in Neural Information Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan and R. Garnett, eds.) 30. Curran Associates, Inc.

95.

Werner, K., Jansson, M. and Stoica, P. (2008). On estimation of covariance matrices with Kronecker product structure. IEEE Transactions on Signal Processing 56 478–491.Werner, K., Jansson, M. and Stoica, P. (2008). On estimation of covariance matrices with Kronecker product structure. IEEE Transactions on Signal Processing 56 478–491.

96.

Wiesel, A., Zhang, T. et al. (2015). Structured robust covariance estimation. Foundations and Trends® in Signal Processing 8 127–216.Wiesel, A., Zhang, T. et al. (2015). Structured robust covariance estimation. Foundations and Trends® in Signal Processing 8 127–216.

97.

Xu, P., Zhang, T. and Gu, Q. (2017). Efficient algorithm for sparse tensor-variate gaussian graphical models via gradient descent. In Artificial Intelligence and Statistics 923–932. PMLR.Xu, P., Zhang, T. and Gu, Q. (2017). Efficient algorithm for sparse tensor-variate gaussian graphical models via gradient descent. In Artificial Intelligence and Statistics 923–932. PMLR.

98.

Yin, J. and Li, H. (2012). Model selection and estimation in the matrix normal graphical model. Journal of multivariate analysis 107 119–140.Yin, J. and Li, H. (2012). Model selection and estimation in the matrix normal graphical model. Journal of multivariate analysis 107 119–140.

99.

Yoon, J. H. and Kim, S. (2020). EiGLasso: Scalable estimation of Cartesian product of sparse inverse covariance matrices. In Conference on Uncertainty in Artificial Intelligence 1248–1257. PMLR.Yoon, J. H. and Kim, S. (2020). EiGLasso: Scalable estimation of Cartesian product of sparse inverse covariance matrices. In Conference on Uncertainty in Artificial Intelligence 1248–1257. PMLR.

100.

Yu, K., Bengtsson, M., Ottersten, B., McNamara, D., Karlsson, P. and Beach, M. (2001). Second order statistics of NLOS indoor MIMO channels based on 5.2 GHz measurements. In GLOBECOM’01. IEEE Global Telecommunications Conference (Cat. No. 01CH37270) 1 156–160. IEEE.Yu, K., Bengtsson, M., Ottersten, B., McNamara, D., Karlsson, P. and Beach, M. (2001). Second order statistics of NLOS indoor MIMO channels based on 5.2 GHz measurements. In GLOBECOM’01. IEEE Global Telecommunications Conference (Cat. No. 01CH37270) 1 156–160. IEEE.

101.

Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19–35.Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19–35.

102.

Zhang, Y. and Schneider, J. G. (2010). Learning multiple tasks with a sparse matrix-normal penalty. In NIPS 6 2.Zhang, Y. and Schneider, J. G. (2010). Learning multiple tasks with a sparse matrix-normal penalty. In NIPS 6 2.

103.

Zhang, A. and Xia, D. (2018). Tensor SVD: Statistical and computational limits. IEEE Transactions on Information Theory 64 7311–7338.Zhang, A. and Xia, D. (2018). Tensor SVD: Statistical and computational limits. IEEE Transactions on Information Theory 64 7311–7338.

104.

Zhou, S. (2014). Gemini: Graph estimation with matrix variate normal instances. The Annals of Statistics 42 532–562.Zhou, S. (2014). Gemini: Graph estimation with matrix variate normal instances. The Annals of Statistics 42 532–562.

105.

Zhou, H., Li, L. and Zhu, H. (2013). Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association 108 540–552.  MR3174640Zhou, H., Li, L. and Zhu, H. (2013). Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association 108 540–552.  MR3174640

106.

Zhou, J., Sun, W. W., Zhang, J. and Li, L. (2021). Partially observed dynamic tensor response regression. Journal of the American Statistical Association 1–16.Zhou, J., Sun, W. W., Zhang, J. and Li, L. (2021). Partially observed dynamic tensor response regression. Journal of the American Statistical Association 1–16.
Yu Wang, Zeyu Sun, Dogyoon Song, and Alfred Hero "Kronecker-structured covariance models for multiway data," Statistics Surveys 16(none), 238-270, (2022). https://doi.org/10.1214/22-SS139
Received: 1 January 2022; Published: 2022
Vol.16 • 2022
Back to Top