Statistical Science

Fourth Moments and Independent Component Analysis

Jari Miettinen, Sara Taskinen, Klaus Nordhausen, and Hannu Oja

Full-text: Open access

Abstract

In independent component analysis it is assumed that the components of the observed random vector are linear combinations of latent independent random variables, and the aim is then to find an estimate for a transformation matrix back to these independent components. In the engineering literature, there are several traditional estimation procedures based on the use of fourth moments, such as FOBI (fourth order blind identification), JADE (joint approximate diagonalization of eigenmatrices), and FastICA, but the statistical properties of these estimates are not well known. In this paper various independent component functionals based on the fourth moments are discussed in detail, starting with the corresponding optimization problems, deriving the estimating equations and estimation algorithms, and finding asymptotic statistical properties of the estimates. Comparisons of the asymptotic variances of the estimates in wide independent component models show that in most cases JADE and the symmetric version of FastICA perform better than their competitors.

Article information

Source
Statist. Sci., Volume 30, Number 3 (2015), 372-390.

Dates
First available in Project Euclid: 10 August 2015

Permanent link to this document
https://projecteuclid.org/euclid.ss/1439220718

Digital Object Identifier
doi:10.1214/15-STS520

Mathematical Reviews number (MathSciNet)
MR3383886

Zentralblatt MATH identifier
1332.62196

Keywords
Affine equivariance FastICA FOBI JADE kurtosis

Citation

Miettinen, Jari; Taskinen, Sara; Nordhausen, Klaus; Oja, Hannu. Fourth Moments and Independent Component Analysis. Statist. Sci. 30 (2015), no. 3, 372--390. doi:10.1214/15-STS520. https://projecteuclid.org/euclid.ss/1439220718


Export citation

References

  • Bonhomme, S. and Robin, J.-M. (2009). Consistent noisy independent component analysis. J. Econometrics 149 12–25.
  • Brys, G., Hubert, M. and Struyf, A. (2006). Robust measures of tail weight. Comput. Statist. Data Anal. 50 733–759.
  • Bugrien, J. B. and Kent, J. T. (2005). Independent component analysis: An approach to clustering. In Proceedings in Quantitative Biology, Shape Analysis and Wavelets (S. Barber, P. D. Baxter, K. V. Mardia and R. E. Walls, eds.) 111–114. Leeds Univ. Press, Leeds, UK.
  • Cardoso, J. F. (1989). Source separation using higher order moments. In Proc. IEEE International Conference on Accoustics, Speech and Signal Processing 2109–2112, Glasgow, UK.
  • Cardoso, J. F. and Souloumiac, A. (1993). Blind beamforming for non Gaussian signals. IEE Proc. F 140 362–370.
  • Caussinus, H. and Ruiz-Gazen, A. (1993). Projection pursuit and generalized principal component analyses. In New Directions in Statistical Data Analysis and Robustness (Ascona, 1992). Monte Verità 35–46. Birkhäuser, Basel.
  • Chen, A. and Bickel, P. J. (2006). Efficient independent component analysis. Ann. Statist. 34 2825–2855.
  • Clarkson, D. B. (1988). A least squares version of algorithm AS 211: The F-G diagonalization algorithm. Appl. Stat. 37 317–321.
  • Critchley, F., Pires, A. and Amado, C. (2006). Principal axis analysis. Technical Report 06/14, The Open Univ., Milton Keynes, UK.
  • Darlington, R. B. (1970). Is kurtosis really “peakedness?” Amer. Statist. 24 19–22.
  • DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychol. Methods 2 292–307.
  • Eriksson, J. and Koivunen, V. (2004). Identifiability, separability and uniqueness of linear ICA models. IEEE Signal Process. Lett. 11 601–604.
  • Friedman, J. H. and Tukey, J. W. (1974). A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. C 23 881–890.
  • Hallin, M. and Mehta, C. (2015). $R$-estimation for asymmetric independent component analysis. J. Amer. Statist. Assoc. 110 218–232.
  • Huber, P. J. (1981). Robust Statistics. Wiley, New York.
  • Huber, P. J. (1985). Projection pursuit. Ann. Statist. 13 435–525.
  • Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10 626–634.
  • Hyvärinen, A., Karhunen, J. and Oja, E. (2001). Independent Component Analysis. Wiley, New York.
  • Hyvärinen, A. and Oja, E. (1997). A fast fixed-point algorithm for independent component analysis. Neural Comput. 9 1483–1492.
  • Ilmonen, P., Nevalainen, J. and Oja, H. (2010). Characteristics of multivariate distributions and the invariant coordinate system. Statist. Probab. Lett. 80 1844–1853.
  • Ilmonen, P. and Paindaveine, D. (2011). Semiparametrically efficient inference based on signed ranks in symmetric independent component models. Ann. Statist. 39 2448–2476.
  • Jones, M. C. and Sibson, R. (1987). What is projection pursuit? J. Roy. Statist. Soc. Ser. A 150 1–36.
  • Kankainen, A., Taskinen, S. and Oja, H. (2007). Tests of multinormality based on location vectors and scatter matrices. Stat. Methods Appl. 16 357–379.
  • Karvanen, J. and Koivunen, V. (2002). Blind separation methods based on pearson system and its extensions. Signal Process. 82 663–673.
  • Koldovský, Z., Tichavský, P. and Oja, E. (2006). Efficient variant of algorithm FastICA for independent component analysis attaining the Cramér–Rao lower bound. IEEE Trans. Neural Netw. 17 1265–1277.
  • Kollo, T. (2008). Multivariate skewness and kurtosis measures with an application in ICA. J. Multivariate Anal. 99 2328–2338.
  • Kollo, T. and Srivastava, M. S. (2004). Estimation and testing of parameters in multivariate Laplace distribution. Comm. Statist. Theory Methods 33 2363–2387.
  • Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika 57 519–530.
  • Maronna, R. A. (1976). Robust $M$-estimators of multivariate location and scatter. Ann. Statist. 4 51–67.
  • Miettinen, J., Nordhausen, K., Oja, H. and Taskinen, S. (2013). Fast equivariant JADE. In Proc. 38th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2013) 6153–6157. Vancouver, BC.
  • Miettinen, J., Nordhausen, K., Oja, H. and Taskinen, S. (2014a). Deflation-based FastICA with adaptive choices of nonlinearities. IEEE Trans. Signal Process. 62 5716–5724.
  • Miettinen, J., Illner, K., Nordhausen, K., Oja, H., Taskinen, S. and Theis, F. J. (2014b). Separation of uncorrelated stationary time series using autocovariance matrices. Available at arXiv:1405.3388.
  • Móri, T. F., Rohatgi, V. K. and Székely, G. J. (1993). On multivariate skewness and kurtosis. Theory Probab. Appl. 38 547–551.
  • Nordhausen, K., Oja, H. and Ollila, E. (2011). Multivariate models and the first four moments. In Nonparametric Statistics and Mixture Models 267–287. World Scientific, Singapore.
  • Nordhausen, K., Ilmonen, P., Mandal, A., Oja, H. and Ollila, E. (2011). Deflation-based FastICA reloaded. In Proc. 19th European Signal Processing Conference 2011 (EUSIPCO 2011) 1854–1858. World Scientific, Singapore.
  • Oja, H. (1981). On location, scale, skewness and kurtosis of univariate distributions. Scand. J. Stat. 8 154–168.
  • Oja, H., Sirkiä, S. and Eriksson, J. (2006). Scatter matrices and independent component analysis. Aust. J. Stat. 35 175–189.
  • Ollila, E. (2010). The deflation-based FastICA estimator: Statistical analysis revisited. IEEE Trans. Signal Process. 58 1527–1541.
  • Pearson, K. (1895). Contributions to the mathematical theory of evolution, II: Skew variation in homogeneous material. Philos. Trans. R. Soc. 186 343–414.
  • Pearson, K. (1905). Das Fehlergesetz und seine Verallgemeinerungen durch Fechner und Pearson. A Rejoinder. Biometrika 4 169–212.
  • Peña, D. and Prieto, F. J. (2001). Cluster identification using projections. J. Amer. Statist. Assoc. 96 1433–1445.
  • Peña, D., Prieto, F. J. and Viladomat, J. (2010). Eigenvectors of a kurtosis matrix as interesting directions to reveal cluster structure. J. Multivariate Anal. 101 1995–2007.
  • Samworth, R. J. and Yuan, M. (2012). Independent component analysis via nonparametric maximum likelihood estimation. Ann. Statist. 40 2973–3002.
  • Serfling, R. J. (1980). Approximation Theorems of Mathematical Statistics. Wiley, New York.
  • Tichavsky, P., Koldovsky, Z. and Oja, E. (2006). Performance analysis of the FastICA algorithm and Cramer–Rao bounds for linear independent component analysis. IEEE Trans. Signal Process. 54 1189–1203.
  • Tyler, D. E., Critchley, F., Dümbgen, L. and Oja, H. (2009). Invariant co-ordinate selection. J. R. Stat. Soc. Ser. B Stat. Methodol. 71 549–592.
  • Van Zwet, W. R. (1964). Convex Transformations of Random Variables. Mathematical Centre Tracts 7. Mathematical Centre, Amsterdam.