## The Annals of Statistics

### Statistics on the Stiefel manifold: Theory and applications

#### Abstract

A Stiefel manifold of the compact type is often encountered in many fields of engineering including, signal and image processing, machine learning, numerical optimization and others. The Stiefel manifold is a Riemannian homogeneous space but not a symmetric space. In previous work, researchers have defined probability distributions on symmetric spaces and performed statistical analysis of data residing in these spaces. In this paper, we present original work involving definition of Gaussian distributions on a homogeneous space and show that the maximum-likelihood estimate of the location parameter of a Gaussian distribution on the homogeneous space yields the Fréchet mean (FM) of the samples drawn from this distribution. Further, we present an algorithm to sample from the Gaussian distribution on the Stiefel manifold and recursively compute the FM of these samples. We also prove the weak consistency of this recursive FM estimator. Several synthetic and real data experiments are then presented, demonstrating the superior computational performance of this estimator over the gradient descent based nonrecursive counter part as well as the stochastic gradient descent based method prevalent in literature.

#### Article information

Source
Ann. Statist., Volume 47, Number 1 (2019), 415-438.

Dates
Revised: February 2018
First available in Project Euclid: 30 November 2018

https://projecteuclid.org/euclid.aos/1543568593

Digital Object Identifier
doi:10.1214/18-AOS1692

Mathematical Reviews number (MathSciNet)
MR3909937

Zentralblatt MATH identifier
07036206

Subjects
Primary: 62F12: Asymptotic properties of estimators
Secondary: 58A99: None of the above, but in this section

#### Citation

Chakraborty, Rudrasis; Vemuri, Baba C. Statistics on the Stiefel manifold: Theory and applications. Ann. Statist. 47 (2019), no. 1, 415--438. doi:10.1214/18-AOS1692. https://projecteuclid.org/euclid.aos/1543568593

#### References

• Absil, P.-A., Mahony, R. and Sepulchre, R. (2004). Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Appl. Math. 80 199–220.
• Afsari, B. (2011). Riemannian $L^{p}$ center of mass: Existence, uniqueness, and convexity. Proc. Amer. Math. Soc. 139 655–673.
• Ando, T., Li, C.-K. and Mathias, R. (2004). Geometric means. Linear Algebra Appl. 385 305–334.
• Arnaudon, M., Barbaresco, F. and Yang, L. (2013). Riemannian medians and means with applications to radar signal processing. IEEE J. Sel. Top. Signal Process. 7 595–604.
• Bhatia, R. (1997). Matrix Analysis. Graduate Texts in Mathematics 169. Springer, New York.
• Bhattacharya, A. and Bhattacharya, R. (2008). Statistics on Riemannian manifolds: Asymptotic distribution and curvature. Proc. Amer. Math. Soc. 136 2959–2967.
• Bonnabel, S. (2013). Stochastic gradient descent on Riemannian manifolds. IEEE Trans. Automat. Control 58 2217–2229.
• Cetingul, H. E. and Vidal, R. (2009). Intrinsic mean shift for clustering on Stiefel and Grassmann manifolds. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on 1896–1902. IEEE, NEw York.
• Chakraborty, R., Banerjee, M. and Vemuri, B. (2017). Statistics on the space of trajectories for longitudinal data analysis. In IEEE International Symposium on Biomedical Imaging.
• Chakraborty, R. and Vemuri, B. C. (2015). Recursive Frechet mean computation on the Grassmannian and its applications to computer vision. In The IEEE International Conference on Computer Vision (ICCV).
• Charfi, M., Chebbi, Z., Moakher, M. and Vemuri, B. C. (2013). Bhattacharyya median of symmetric positive-definite matrices and application to the denoising of diffusion-tensor fields. In Biomedical Imaging (ISBI), 2013 IEEE 10th International Symposium on 1227–1230. IEEE, New York.
• Cheeger, J. and Ebin, D. G. (1975). Comparison Theorems in Riemannian Geometry. North-Holland Mathematical Library 9. North-Holland, Amsterdam.
• Cheng, G. and Vemuri, B. C. (2013). A novel dynamic system in the space of SPD matrices with applications to appearance tracking. SIAM J. Imaging Sci. 6 592–615.
• Chikuse, Y. (1991). Asymptotic expansions for distributions of the large sample matrix resultant and related statistics on the Stiefel manifold. J. Multivariate Anal. 39 270–283.
• Cramér, H. (1946). Mathematical Methods of Statistics. Princeton Mathematical Series 9. Princeton Univ. Press, Princeton, NJ.
• Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection. In CVPR 1 886–893.
• Doretto, G., Chiuso, A., Wu, Y. N. and Soatto, S. (2003). Dynamic textures. Int. J. Comput. Vis. 51 91–109.
• Downs, T., Liebman, J. and Mackay, W. (1971). Statistical methods for vectorcardiogram orientations. In Vectorcardiography 2: Proc. XIth International Symp. on Vectorcardiography.
• Fletcher, P. T. (2013). Geodesic regression and the theory of least squares on Riemannian manifolds. Int. J. Comput. Vis. 105 171–185.
• Fletcher, P. T. and Joshi, S. (2007). Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Process. 87 250–262.
• Fletcher, P. T., Lu, C., Pizer, S. M. and Joshi, S. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Trans. Med. Imag. 23 995–1005.
• Fraikin, C., Hüper, K. and Dooren, P. V. (2007). Optimization over the Stiefel manifold. PAMM 7 1062205–1062206.
• Fréchet, M. (1948). Les éléments aléatoires de nature quelconque dans un espace distancié. Ann. Inst. H. Poincaré 10 215–310.
• Goodall, C. R. and Mardia, K. V. (1999). Projective shape analysis. J. Comput. Graph. Statist. 8 143–168.
• Grenander, U. (2008). Probabilities on Algebraic Structures. Dover Publications, Mineola, NY.
• Groisser, D. (2004). Newton’s method, zeroes of vector fields, and the Riemannian center of mass. Adv. in Appl. Math. 33 95–135.
• Hartley, R., Trumpf, J., Dai, Y. and Li, H. (2013). Rotation averaging. Int. J. Comput. Vis. 103 267–305.
• Hauberg, S., Feragen, A. and Black, M. J. (2014). Grassmann averages for scalable robust PCA. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3810–3817.
• Helgason, S. (1978). Differential Geometry, Lie Groups, and Symmetric Spaces. Pure and Applied Mathematics 80. Academic Press, New York.
• Hendriks, H. and Landsman, Z. (1998). Mean location and sample mean location on manifolds: Asymptotics, tests, confidence regions. J. Multivariate Anal. 67 227–243.
• Ho, J., Cheng, G., Salehian, H. and Vemuri, B. (2013). Recursive Karcher expectation estimators and geometric law of large numbers. In Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics 325–332.
• Kaneko, T., Fiori, S. and Tanaka, T. (2013). Empirical arithmetic averaging over the compact Stiefel manifold. IEEE Trans. Signal Process. 61 883–894.
• Karcher, H. (1977). Riemannian center of mass and mollifier smoothing. Comm. Pure Appl. Math. 30 509–541.
• Kendall, W. S. (1990). Probability, convexity, and harmonic maps with small image. I. Uniqueness and fine existence. Proc. Lond. Math. Soc. (3) 61 371–406.
• Lui, Y. M. (2012). Advances in matrix manifolds for computer vision. Image Vis. Comput. 30 380–388.
• Mardia, K. V. and Jupp, P. E. (2000). Directional Statistics. Wiley, Chichester. Revised reprint of Statistics of directional data by Mardia [ MR0336854 (49 #1627)].
• Moakher, M. (2005). A differential geometric approach to the geometric mean of symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 26 735–747.
• Moakher, M. (2006). On the averaging of symmetric positive-definite tensors. J. Elasticity 82 273–296.
• Patrangenaru, V. and Mardia, K. V. (2003). Affine shape analysis and image analysis. In 22nd Leeds Annual Statistics Research Workshop.
• Pennec, X. (2006). Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measurements. J. Math. Imaging Vision 25 127–154.
• Pennec, X., Fillard, P. and Ayache, N. (2006). A Riemannian framework for tensor computing. Int. J. Comput. Vis. 66 41–66.
• Pham, D.-S. and Venkatesh, S. (2008). Robust learning of discriminative projection for multicategory classification on the Stiefel manifold. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on 1–7. IEEE, New York.
• Rao, C. R. (1987). Differential metrics in probability spaces. Differential Geometry in Statistical Inference 10 217–240.
• Rao, C. R. (1992). Information and the accuracy attainable in the estimation of statistical parameters. In Breakthroughs in Statistics 235–247. Springer, New York.
• Said, S., Hajri, H., Bombrun, L. and Vemuri, B. C. (2016). Gaussian distributions on Riemannian symmetric spaces: Statistical learning with structured covariance matrices. ArXiv Preprint ArXiv:1607.06929.
• Said, S., Bombrun, L., Berthoumieu, Y. and Manton, J. H. (2017). Riemannian Gaussian distributions on the space of symmetric positive definite matrices. IEEE Trans. Inform. Theory 63 2153–2170.
• Salehian, H., Chakraborty, R., Ofori, E., Vaillancourt, D. and Vemuri, B. C. (2015). An efficient recursive estimator of the Fréchet mean on a hypersphere with applications to medical image analysis. In Mathematical Foundations of Computational Anatomy.
• Schuldt, C., Laptev, I. and Caputo, B. (2004). Recognizing human actions: A local SVM approach. In Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on 3 32–36.
• Srivastava, A., Jermyn, I. and Joshi, S. (2007). Riemannian analysis of probability density functions with applications in vision. In CVPR 1–8.
• Sturm, K.-T. (2003). Probability measures on metric spaces of nonpositive curvature. In Heat Kernels and Analysis on Manifolds, Graphs, and Metric Spaces (Paris, 2002). Contemp. Math. 338 357–390. Amer. Math. Soc., Providence, RI.
• Tuch, D. S., Reese, T. G., Wiegell, M. R. and Wedeen, V. J. (2003). Diffusion MRI of complex neural architecture. Neuron 40 885–895.
• Turaga, P., Veeraraghavan, A. and Chellappa, R. (2008). Statistical analysis on Stiefel and Grassmann manifolds with applications in computer vision. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on 1–8. IEEE, New York.
• Wong, Y. (1968). Sectional curvatures of Grassmann manifolds. Proc. Natl. Acad. Sci. USA 60 75–79.
• Zhang, M. and Fletcher, P. T. (2013). Probabilistic principal geodesic analysis. In Advances in Neural Information Processing Systems 1178–1186.
• Ziller, W. (2007). Examples of Riemannian manifolds with non-negative sectional curvature. In Surveys in Differential Geometry. Vol. XI. Surv. Differ. Geom. 11 63–102. Int. Press, Somerville, MA.
• Zimmermann, R. (2017). A matrix-algebraic algorithm for the Riemannian logarithm on the Stiefel manifold under the canonical metric. SIAM J. Matrix Anal. Appl. 38 322–342.