Statistical Science
- Statist. Sci.
- Volume 30, Number 3 (2015), 372-390.
Fourth Moments and Independent Component Analysis
Jari Miettinen, Sara Taskinen, Klaus Nordhausen, and Hannu Oja
Full-text: Open access
Abstract
In independent component analysis it is assumed that the components of the observed random vector are linear combinations of latent independent random variables, and the aim is then to find an estimate for a transformation matrix back to these independent components. In the engineering literature, there are several traditional estimation procedures based on the use of fourth moments, such as FOBI (fourth order blind identification), JADE (joint approximate diagonalization of eigenmatrices), and FastICA, but the statistical properties of these estimates are not well known. In this paper various independent component functionals based on the fourth moments are discussed in detail, starting with the corresponding optimization problems, deriving the estimating equations and estimation algorithms, and finding asymptotic statistical properties of the estimates. Comparisons of the asymptotic variances of the estimates in wide independent component models show that in most cases JADE and the symmetric version of FastICA perform better than their competitors.
Article information
Source
Statist. Sci., Volume 30, Number 3 (2015), 372-390.
Dates
First available in Project Euclid: 10 August 2015
Permanent link to this document
https://projecteuclid.org/euclid.ss/1439220718
Digital Object Identifier
doi:10.1214/15-STS520
Mathematical Reviews number (MathSciNet)
MR3383886
Zentralblatt MATH identifier
1332.62196
Keywords
Affine equivariance FastICA FOBI JADE kurtosis
Citation
Miettinen, Jari; Taskinen, Sara; Nordhausen, Klaus; Oja, Hannu. Fourth Moments and Independent Component Analysis. Statist. Sci. 30 (2015), no. 3, 372--390. doi:10.1214/15-STS520. https://projecteuclid.org/euclid.ss/1439220718
References
- Bonhomme, S. and Robin, J.-M. (2009). Consistent noisy independent component analysis. J. Econometrics 149 12–25.Mathematical Reviews (MathSciNet): MR2515042
Digital Object Identifier: doi:10.1016/j.jeconom.2008.12.019 - Brys, G., Hubert, M. and Struyf, A. (2006). Robust measures of tail weight. Comput. Statist. Data Anal. 50 733–759.Mathematical Reviews (MathSciNet): MR2207005
- Bugrien, J. B. and Kent, J. T. (2005). Independent component analysis: An approach to clustering. In Proceedings in Quantitative Biology, Shape Analysis and Wavelets (S. Barber, P. D. Baxter, K. V. Mardia and R. E. Walls, eds.) 111–114. Leeds Univ. Press, Leeds, UK.
- Cardoso, J. F. (1989). Source separation using higher order moments. In Proc. IEEE International Conference on Accoustics, Speech and Signal Processing 2109–2112, Glasgow, UK.
- Cardoso, J. F. and Souloumiac, A. (1993). Blind beamforming for non Gaussian signals. IEE Proc. F 140 362–370.
- Caussinus, H. and Ruiz-Gazen, A. (1993). Projection pursuit and generalized principal component analyses. In New Directions in Statistical Data Analysis and Robustness (Ascona, 1992). Monte Verità 35–46. Birkhäuser, Basel.
- Chen, A. and Bickel, P. J. (2006). Efficient independent component analysis. Ann. Statist. 34 2825–2855.Mathematical Reviews (MathSciNet): MR2329469
Zentralblatt MATH: 1114.62033
Digital Object Identifier: doi:10.1214/009053606000000939
Project Euclid: euclid.aos/1179935066 - Clarkson, D. B. (1988). A least squares version of algorithm AS 211: The F-G diagonalization algorithm. Appl. Stat. 37 317–321.
- Critchley, F., Pires, A. and Amado, C. (2006). Principal axis analysis. Technical Report 06/14, The Open Univ., Milton Keynes, UK.
- Darlington, R. B. (1970). Is kurtosis really “peakedness?” Amer. Statist. 24 19–22.
- DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychol. Methods 2 292–307.
- Eriksson, J. and Koivunen, V. (2004). Identifiability, separability and uniqueness of linear ICA models. IEEE Signal Process. Lett. 11 601–604.
- Friedman, J. H. and Tukey, J. W. (1974). A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. C 23 881–890.
- Hallin, M. and Mehta, C. (2015). $R$-estimation for asymmetric independent component analysis. J. Amer. Statist. Assoc. 110 218–232.Mathematical Reviews (MathSciNet): MR3338498
Digital Object Identifier: doi:10.1080/01621459.2014.909316 - Huber, P. J. (1981). Robust Statistics. Wiley, New York.Mathematical Reviews (MathSciNet): MR606374
- Huber, P. J. (1985). Projection pursuit. Ann. Statist. 13 435–525.Mathematical Reviews (MathSciNet): MR790553
Zentralblatt MATH: 0595.62059
Digital Object Identifier: doi:10.1214/aos/1176349519
Project Euclid: euclid.aos/1176349519 - Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10 626–634.
- Hyvärinen, A., Karhunen, J. and Oja, E. (2001). Independent Component Analysis. Wiley, New York.
- Hyvärinen, A. and Oja, E. (1997). A fast fixed-point algorithm for independent component analysis. Neural Comput. 9 1483–1492.
- Ilmonen, P., Nevalainen, J. and Oja, H. (2010). Characteristics of multivariate distributions and the invariant coordinate system. Statist. Probab. Lett. 80 1844–1853.Mathematical Reviews (MathSciNet): MR2734250
- Ilmonen, P. and Paindaveine, D. (2011). Semiparametrically efficient inference based on signed ranks in symmetric independent component models. Ann. Statist. 39 2448–2476.Mathematical Reviews (MathSciNet): MR2906874
Zentralblatt MATH: 1231.62043
Digital Object Identifier: doi:10.1214/11-AOS906
Project Euclid: euclid.aos/1322663464 - Jones, M. C. and Sibson, R. (1987). What is projection pursuit? J. Roy. Statist. Soc. Ser. A 150 1–36.
- Kankainen, A., Taskinen, S. and Oja, H. (2007). Tests of multinormality based on location vectors and scatter matrices. Stat. Methods Appl. 16 357–379.Mathematical Reviews (MathSciNet): MR2399867
- Karvanen, J. and Koivunen, V. (2002). Blind separation methods based on pearson system and its extensions. Signal Process. 82 663–673.
- Koldovský, Z., Tichavský, P. and Oja, E. (2006). Efficient variant of algorithm FastICA for independent component analysis attaining the Cramér–Rao lower bound. IEEE Trans. Neural Netw. 17 1265–1277.
- Kollo, T. (2008). Multivariate skewness and kurtosis measures with an application in ICA. J. Multivariate Anal. 99 2328–2338.Mathematical Reviews (MathSciNet): MR2463392
Zentralblatt MATH: 1294.62021
Digital Object Identifier: doi:10.1016/j.jmva.2008.02.033 - Kollo, T. and Srivastava, M. S. (2004). Estimation and testing of parameters in multivariate Laplace distribution. Comm. Statist. Theory Methods 33 2363–2387.Mathematical Reviews (MathSciNet): MR2104118
Zentralblatt MATH: 1217.62080
Digital Object Identifier: doi:10.1081/STA-200031408 - Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika 57 519–530.Mathematical Reviews (MathSciNet): MR397994
Zentralblatt MATH: 0214.46302
Digital Object Identifier: doi:10.1093/biomet/57.3.519 - Maronna, R. A. (1976). Robust $M$-estimators of multivariate location and scatter. Ann. Statist. 4 51–67.Mathematical Reviews (MathSciNet): MR388656
Zentralblatt MATH: 0322.62054
Digital Object Identifier: doi:10.1214/aos/1176343347
Project Euclid: euclid.aos/1176343347 - Miettinen, J., Nordhausen, K., Oja, H. and Taskinen, S. (2013). Fast equivariant JADE. In Proc. 38th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013) 6153–6157. Vancouver, BC.
- Miettinen, J., Nordhausen, K., Oja, H. and Taskinen, S. (2014a). Deflation-based FastICA with adaptive choices of nonlinearities. IEEE Trans. Signal Process. 62 5716–5724.Mathematical Reviews (MathSciNet): MR3273526
Digital Object Identifier: doi:10.1109/TSP.2014.2356442 - Miettinen, J., Illner, K., Nordhausen, K., Oja, H., Taskinen, S. and Theis, F. J. (2014b). Separation of uncorrelated stationary time series using autocovariance matrices. Available at arXiv:1405.3388.arXiv: 1405.3388
- Móri, T. F., Rohatgi, V. K. and Székely, G. J. (1993). On multivariate skewness and kurtosis. Theory Probab. Appl. 38 547–551.Mathematical Reviews (MathSciNet): MR1404679
- Nordhausen, K., Oja, H. and Ollila, E. (2011). Multivariate models and the first four moments. In Nonparametric Statistics and Mixture Models 267–287. World Scientific, Singapore.Mathematical Reviews (MathSciNet): MR2838731
Digital Object Identifier: doi:10.1142/9789814340564_0016 - Nordhausen, K., Ilmonen, P., Mandal, A., Oja, H. and Ollila, E. (2011). Deflation-based FastICA reloaded. In Proc. 19th European Signal Processing Conference 2011 (EUSIPCO 2011) 1854–1858. World Scientific, Singapore.
- Oja, H. (1981). On location, scale, skewness and kurtosis of univariate distributions. Scand. J. Stat. 8 154–168.Mathematical Reviews (MathSciNet): MR633040
- Oja, H., Sirkiä, S. and Eriksson, J. (2006). Scatter matrices and independent component analysis. Aust. J. Stat. 35 175–189.
- Ollila, E. (2010). The deflation-based FastICA estimator: Statistical analysis revisited. IEEE Trans. Signal Process. 58 1527–1541.Mathematical Reviews (MathSciNet): MR2758026
Digital Object Identifier: doi:10.1109/TSP.2009.2036072 - Pearson, K. (1895). Contributions to the mathematical theory of evolution, II: Skew variation in homogeneous material. Philos. Trans. R. Soc. 186 343–414.
- Pearson, K. (1905). Das Fehlergesetz und seine Verallgemeinerungen durch Fechner und Pearson. A Rejoinder. Biometrika 4 169–212.
- Peña, D. and Prieto, F. J. (2001). Cluster identification using projections. J. Amer. Statist. Assoc. 96 1433–1445.Mathematical Reviews (MathSciNet): MR1946588
Zentralblatt MATH: 1051.62055
Digital Object Identifier: doi:10.1198/016214501753382345 - Peña, D., Prieto, F. J. and Viladomat, J. (2010). Eigenvectors of a kurtosis matrix as interesting directions to reveal cluster structure. J. Multivariate Anal. 101 1995–2007.Mathematical Reviews (MathSciNet): MR2671197
Zentralblatt MATH: 1203.62114
Digital Object Identifier: doi:10.1016/j.jmva.2010.04.014 - Samworth, R. J. and Yuan, M. (2012). Independent component analysis via nonparametric maximum likelihood estimation. Ann. Statist. 40 2973–3002.Mathematical Reviews (MathSciNet): MR3097966
Zentralblatt MATH: 1296.62084
Digital Object Identifier: doi:10.1214/12-AOS1060
Project Euclid: euclid.aos/1360332190 - Serfling, R. J. (1980). Approximation Theorems of Mathematical Statistics. Wiley, New York.Mathematical Reviews (MathSciNet): MR595165
- Tichavsky, P., Koldovsky, Z. and Oja, E. (2006). Performance analysis of the FastICA algorithm and Cramer–Rao bounds for linear independent component analysis. IEEE Trans. Signal Process. 54 1189–1203.
- Tyler, D. E., Critchley, F., Dümbgen, L. and Oja, H. (2009). Invariant co-ordinate selection. J. R. Stat. Soc. Ser. B Stat. Methodol. 71 549–592.Mathematical Reviews (MathSciNet): MR2749907
Zentralblatt MATH: 1250.62032
Digital Object Identifier: doi:10.1111/j.1467-9868.2009.00706.x - Van Zwet, W. R. (1964). Convex Transformations of Random Variables. Mathematical Centre Tracts 7. Mathematical Centre, Amsterdam.Mathematical Reviews (MathSciNet): MR176511
- You have access to this content.
- You have partial access to this content.
- You do not have access to this content.
More like this
- Efficient independent component analysis
Chen, Aiyou and Bickel, Peter J., The Annals of Statistics, 2006 - Estimating multivariate latent-structure models
Bonhomme, Stéphane, Jochmans, Koen, and Robin, Jean-Marc, The Annals of Statistics, 2016 - The Asymptotic Normal Distribution of Estimators in Factor Analysis under General Conditions
Anderson, T. W. and Amemiya, Yasuo, The Annals of Statistics, 1988
- Efficient independent component analysis
Chen, Aiyou and Bickel, Peter J., The Annals of Statistics, 2006 - Estimating multivariate latent-structure models
Bonhomme, Stéphane, Jochmans, Koen, and Robin, Jean-Marc, The Annals of Statistics, 2016 - The Asymptotic Normal Distribution of Estimators in Factor Analysis under General Conditions
Anderson, T. W. and Amemiya, Yasuo, The Annals of Statistics, 1988 - A Multivariate Variance Components Model for Analysis of Covariance in Designed Experiments
Booth, James G., Federer, Walter T., Wells, Martin T., and Wolfinger, Russell D., Statistical Science, 2009 - Bayesian nonparametric disclosure risk estimation via mixed effects log-linear models
Carota, Cinzia, Filippone, Maurizio, Leombruni, Roberto, and Polettini, Silvia, The Annals of Applied Statistics, 2015 - Minimax estimation with thresholding and its application to wavelet analysis
Zhou, Harrison H. and Hwang, J. T. Gene, The Annals of Statistics, 2005 - Rate-adaptive Bayesian independent component analysis
Shen, Weining, Ning, Jing, and Yuan, Ying, Electronic Journal of Statistics, 2016 - Semiparametrically efficient inference based on signed ranks in symmetric independent component models
Ilmonen, Pauliina and Paindaveine, Davy, The Annals of Statistics, 2011 - Multilayer tensor factorization with applications to recommender systems
Bi, Xuan, Qu, Annie, and Shen, Xiaotong, The Annals of Statistics, 2018 - Latent variable graphical model selection via convex optimization
Chandrasekaran, Venkat, Parrilo, Pablo A., and Willsky, Alan S., The Annals of Statistics, 2012