The Annals of Statistics

Independent component analysis via nonparametric maximum likelihood estimation

Richard J. Samworth and Ming Yuan

Full-text: Open access

Abstract

Independent Component Analysis (ICA) models are very popular semiparametric models in which we observe independent copies of a random vector $X=AS$, where $A$ is a non-singular matrix and $S$ has independent components. We propose a new way of estimating the unmixing matrix $W=A^{-1}$ and the marginal distributions of the components of $S$ using nonparametric maximum likelihood. Specifically, we study the projection of the empirical distribution onto the subset of ICA distributions having log-concave marginals. We show that, from the point of view of estimating the unmixing matrix, it makes no difference whether or not the log-concavity is correctly specified. The approach is further justified by both theoretical results and a simulation study.

Article information

Source
Ann. Statist., Volume 40, Number 6 (2012), 2973-3002.

Dates
First available in Project Euclid: 8 February 2013

Permanent link to this document
https://projecteuclid.org/euclid.aos/1360332190

Digital Object Identifier
doi:10.1214/12-AOS1060

Mathematical Reviews number (MathSciNet)
MR3097966

Zentralblatt MATH identifier
1296.62084

Subjects
Primary: 62G07: Density estimation

Keywords
Blind source separation density estimation independent component analysis log-concave projection nonparametric maximum likelihood estimator

Citation

Samworth, Richard J.; Yuan, Ming. Independent component analysis via nonparametric maximum likelihood estimation. Ann. Statist. 40 (2012), no. 6, 2973--3002. doi:10.1214/12-AOS1060. https://projecteuclid.org/euclid.aos/1360332190


Export citation

References

  • Bach, F. R. and Jordan, M. I. (2002). Kernel independent component analysis. J. Mach. Learn. Res. 3 1–48.
  • Chen, A. and Bickel, P. J. (2005). Consistent independent component analysis and prewhitening. IEEE Trans. Signal Process. 53 3625–3632.
  • Chen, A. and Bickel, P. J. (2006). Efficient independent component analysis. Ann. Statist. 34 2825–2855.
  • Chen, Y. and Samworth, R. J. (2012). Smoothed log-concave maximum likelihood estimation with applications. Statist. Sinica. To appear.
  • Comon, P. (1994). Independent component analysis, a new concept? Signal Proc. 36 287–314.
  • Cule, M. and Samworth, R. (2010). Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density. Electron. J. Stat. 4 254–270.
  • Cule, M., Samworth, R. and Stewart, M. (2010). Maximum likelihood estimation of a multi-dimensional log-concave density. J. R. Stat. Soc. Ser. B Stat. Methodol. 72 545–607.
  • Dümbgen, L. and Rufibach, K. (2011). logcondens: Computations related to univariate log-concave density estimation. J. Statist. Software 39 1–28.
  • Dümbgen, L., Samworth, R. and Schuhmacher, D. (2011). Approximation by log-concave distributions, with applications to regression. Ann. Statist. 39 702–730.
  • Eriksson, J. and Koivunen, V. (2004). Identifiability, separability and uniqueness of linear ICA models. IEEE Signal Processing Letters 11 601–604.
  • Hastie, T. and Tibshirani, R. (2003a). ProDenICA: Product density estimation for ICA using tilted Gaussian density estimates R package version 1.0. Available at http://cran.r-project.org/web/packages/ProDenICA/.
  • Hastie, T. and Tibshirani, R. (2003b). Independent component analysis through product density estimation. In Advances in Neural Information Processing Systems 15 (S. Becker and K. Obermayer, eds.) 649–656. MIT Press, Cambridge, MA.
  • Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning. Springer, New York.
  • Hyvärinen, A., Karhunen, J. and Oja, E. (2001). Independent Component Analysis. Wiley, New York.
  • Hyvärinen, A. and Oja, E. (2000). Independent component analysis: Algorithms and applications. Neural Netw. 13 411–430.
  • Ilmonen, P., Nevalainen, J. and Oja, H. (2010). Characteristics of multivariate distributions and the invariant coordinate system. Statist. Probab. Lett. 80 1844–1853.
  • Ilmonen, P. and Paindaveine, D. (2011). Semiparametrically efficient inference based on signed ranks in symmetric independent component models. Ann. Statist. 39 2448–2476.
  • Karvanen, J. and Koivunen, V. (2002). Blind separation methods based on Pearson system and its extensions. Signal Proc. 82 663–673.
  • Nordhausen, K., Oja, H. and Ollila, E. (2011). Multivariate models and the first four moments. In Nonparametric Statistics and Mixture Models 267–287. World Sci. Publ., Hackensack, NJ.
  • Nordhausen, K., Ilmonen, P., Mandal, A., Oja, H. and Ollila, E. (2011). Deflation-based FastICA reloaded. In Proceedings of 19th European Signal Processing Conference 2011 (EUSIPCO 2011) 1854–1858.
  • Oja, H., Sirkiä, S. and Eriksson, J. (2006). Scatter matrices and independent component analysis. Austrian J. Statist. 35 175–189.
  • Ollila, E. (2010). The deflation-based FastICA estimator: Statistical analysis revisited. IEEE Trans. Signal Process. 58 1527–1541.
  • Ollila, E., Oja, H. and Koivunen, V. (2008). Complex-valued ICA based on a pair of generalized covariance matrices. Comput. Statist. Data Anal. 52 3789–3805.
  • Owen, A. (1990). Empirical Likelihood. Chapman & Hall, London.
  • Plumbley, M. D. (2005). Geometrical methods for non-negative ICA: Manifolds, lie groups and toral subalgebras. Neurocomputing 67 161–197.
  • Prékopa, A. (1973). On logarithmic concave measures and functions. Acta Sci. Math. (Szeged) 34 335–343.
  • Rufibach, K. and Dümbgen, L. (2006). logcondens: Estimate a log-concave probability density from i.i.d. observations R package version 2.01. Available at http://cran.r-project.org/web/packages/logcondens/.
  • Samarov, A. and Tsybakov, A. (2004). Nonparametric independent component analysis. Bernoulli 10 565–582.