Statistical Science

Nonlinear Factor Analysis as a Statistical Method

Yasuo Amemiya and Ilker Yalcin

Full-text: Open access


Factor analysis and its extensions are widely used in the social and behavioral sciences, and can be considered useful tools for exploration and model fitting in multivariate analysis. Despite its popularity in applications, factor analysis has attracted rather limited attention from statisticians. Three issues, identification ambiguity, heavy reliance on normality, and limitation to linearity, may have contributed to statisticians' lack of interest in factor analysis. In this paper, the statistical contributions to the first two issues are reviewed, and the third issue is addressed in detail. Linear models can be unrealistic even as an approximation in many applications, and often do not fit the data well without increasing the number of factors beyond the level explainable by the subject-matter theory. As an exploratory model, the conventional factor analysis model fails to address nonlinear structure underlying multivariate data. It is argued here that factor analysis does not need to be restricted to linearity and that nonlinear factor analysis can be formulated and carried out as a useful statistical method. In particular, for a general parametric nonlinear factor analysis model, the errors- in-variables parameterization is suggested as a sensible way to formulate the model, and two procedures for model fitting are introduced and described. Tests for the goodness-of-fit of the model are also proposed. The procedures are studied through a simulation study. An example from personality testing is used to illustrate the issues and the methods.

Article information

Statist. Sci. Volume 16, Issue 3 (2001), 275-294.

First available in Project Euclid: 24 December 2001

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Yalcin, Ilker; Amemiya, Yasuo. Nonlinear Factor Analysis as a Statistical Method. Statistical Science 16 (2001), no. 3, 275--294. doi:10.1214/ss/1009213729.

Export citation


  • Amemiy a, Y. (1985). What should be done when an estimated between-group covariance matrix is not nonnegative definite? Amer. Statist. 39 112-117. Amemiy a, Y. (1993a). Instrumental variable estimation for nonlinear factor analysis. In Multivariate Analy sis: Future Directions 2 (C. M. Cuadras and C. R. Rao, eds.) 113-129. North-Holland, Amsterdam. Amemiy a, Y. (1993b). On nonlinear factor analysis. Proceedings of the ASA Social Statistics Section 290-294.
  • Amemiy a, Y. and Anderson, T. W. (1990). Asy mptotic chi-square tests for a large class of factor analysis models. Ann. Statist. 18 1453-1463.
  • Amemiy a, Y. and Fuller, W. A. (1988). Estimation for the nonlinear functional relationship. Ann. Statist. 16 147-160.
  • Amemiy a, Y., Pantula, S. G. and Fuller, W. A. (1987). The asy mptotic distribution of some estimators for a factor analysis model. J. Multivariate Anal. 22 51-64.
  • Anderson, T. W. and Amemiy a, Y. (1988). The asy mptotic normal distribution of estimators in factor analysis under general conditions. Ann. Statist. 16 759-771.
  • Anderson, T. W. and Rubin, H. (1956). Statistical inference in factor analysis. Proc. Third Berkeley Sy mp. Math. Statist. Probab. 5 111-150. Univ. California Press, Berkeley.
  • Arminger, G. and Muth´en, B. (1998). A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the Metropolis-Hastings algorithm. Psy chometrika 63 271-300.
  • Bartlett, M. S. (1953). Factor analysis in psy chology as a statistician sees it. In Uppsala Sy mposium on Psy chological Factor Analy sis 23-24. Almqvist and Wiksell, Stockholm.
  • Browne, M. W. (1984). Asy mptotically distribution-free methods for the analysis covariance structures. British J. Math. Statist. Psy ch. 37 62-83.
  • Browne, M. W. (1987). Robustness of statistical inference in factor analysis and related models. Biometrika 74 375-384.
  • Browne, M. W. and Shapiro, A. (1988). Robustness of normal theory methods in the analysis of linear latent variate models. British J. Math. Statist. Psy ch. 41 193-208.
  • Carroll, R. J. and Ruppert, D. (1982). A comparison between maximum likelihood and generalized least squares in a heteroscedastic linear model. Amer. Statist. Assoc. 77 878-882.
  • Carroll, R. J. and Ruppert, D. (1988). Transformation and Weighting in Regression. Chapman and Hall, New York.
  • Carroll, R. J., Ruppert, D. and Stefanski, L. A. (1995). Measurement Error in Nonlinear Models. Chapman and Hall, New York.
  • Cheng, C. and Van Ness, J. W. (1999). Statistical Regression with Measurement Error. Arnold, London.
  • Dumenci, L. (1993). Personality from a broad perspective: Hierarchical item-level factor analysis of personality, temperament, and vocational interests. Ph.D. dissertation, Iowa State Univ. Ames, IA.
  • Etezadi-Amoli, J. and Mc Donald, R. P. (1983). A second generation nonlinear factor analysis. Psy chometrika 48 315-342.
  • Fuller, W. A. (1987). Measurement Error Models. Wiley, New York.
  • Gnanadesikan, R. and Kettenring, J. R. (1984). A pragmatic review of multivariate methods in applications. In Statistics: An Appraisal (H. A. David and H. T. David, eds.) 309-337. Iowa State Univ. Press, Ames, IA.
  • Hu, L., Bentler, P. and Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psy chological Bulletin 112 351-362.
  • Jennrich, R. I. and Robinson, S. M. (1969). A Newton-Raphson algorithm for maximum likelihood factor analysis. Psy chometrika 34 111-123.
  • J ¨oreskog, K. G. (1967). Some contributions to maximum likelihood factor analysis. Psy chometrika 32 443-482.
  • J ¨oreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psy chometrika 34 183-202.
  • J ¨oreskog, K. G. (1970). A general method for analysis of covariance structure. Biometrika 57 239-252.
  • J ¨oreskog, K. G. (1973). A general method for estimating a linear structural equation sy stem. In Structural Equation Models in the Social Sciences (A. S. Goldberger and O. D. Duncun, eds.) 85-112. Seminar Press, New York.
  • J ¨oreskog, K. G. (1977). Factor analysis by least squares and maximum likelihood methods. In Statistical Methods for Digital Computers III (K. Enslein, A. Ralston and H. S. Wilf, eds.) 13 125-153. Wiley, New York.
  • J ¨oreskog, K. G. (1994). On the estimation of poly choric correlations and their asy mptotic covariance matrix. Psy chometrika 59 381-389.
  • J ¨oreskog, K. G. and S ¨orbom, D. (1989). LISREL 7: User's Reference Guide. Scientific Software, Mooresville, IN.
  • J ¨oreskog, K. G. and Yang, F. (1994). Non-linear structural equation models: The Kenny-Judd model with interaction effects. In Advanced Structural Equation Modeling: Issues and Techniques (G. A. Marcoulides and R. E. Schumacker, eds.) 57-88. Erlbaum, Mahwah, NJ.
  • J ¨oreskog, K. G. and Yang, F. (1997). Estimation of interaction models using the augmented moment matrix: comparison of asy mptotic standard errors. In SoftStat '97: Advances in Statistical Software (W. Bandilla and F. Faulbaum, eds.) 6 467-478. Lucius, Stuttgart.
  • Kenney, D. A. and PJudd, C. M. (1984). Estimating the nonlinear and interactive effects of latent variables. Psy chological Bulletin 96 201-210.
  • Lawley, D. N. (1940). The estimation of factor loadings by the method of maximum likelihood. Proc. Roy. Soc. Edinburgh Sec. A 60 64-82.
  • Lawley, D. N. (1941). Further investigations in factor estimation. Proc. Roy. Soc. Edinburgh Sec. A 61 176-185.
  • Lawley, D. N. (1943). The application of the maximum likelihood method to factor analysis. British J. Psy chology 33 172-175.
  • Mc Ardle, J. J. and Epstein, D. (1987). Latent growth curves within developmental structural equation models. Child Development 58 110-133.
  • Mc Donald, R. P. (1962). A general approach to nonlinear factor analysis. Psy chometrika 27 397-415.
  • Mc Donald, R. P. (1965). Difficulty factors and nonlinear factor analysis. British J. Math. Statist. Psy ch. 18 11-23. Mc Donald, R. P. (1967a). Numerical methods for poly nomial models in nonlinear factor analysis. Psy chometrika 32 77-112. Mc Donald, R. P. (1967b). Factor interaction in nonlinear factor analysis. British J. Math. Statist. Psy ch. 20 205-215.
  • Mc Donald, R. P. (1979). The simultaneous estimation of factor loadings and scores. British J. Math. Statist. Psy ch. 32 212-228.
  • Meredith, W. and Tisak, J. (1990). Latent curve analysis. Psy chometrika 55 107-122.
  • Mooijaart, A. and Bentler, P. (1986). Random poly nomial factor analysis. In Data Analy sis and Informatics (E. Diday, Y. Escoufier, L. Lebart, J. Pages, Y. Schektman and R. Tomassone, eds.) 4 241-250. North-Holland, Amsterdam.
  • Muth´en, B. (1984). A general structual equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psy chometrika 49 115-132.
  • Pantula, S. G. and Fuller, W. A. (1986). Computational algorithms for the factor model. Comm. Statist. Ser. B 15 227-259.
  • Satorra, A. (1992). Asy mptotic robust inferences in the analysis of mean and covariance structures. Sociological Methodology 22 249-278.
  • Van Houwelingen, J. C. (1988). Use and abuse of variance models in regression. Biometrics 44 1073-1081.
  • Wall, M. M. and Amemiy a, Y. (2000). Estimation for poly nomial structural equation models. J. Amer. Statist. Assoc. 95 929-940.
  • Wall, M. M. and Amemiy a, Y. (2001). Generalized appended product indicator procedure for fitting poly nomial structural models. J. Educational and Behavioral Statist. 26 1-29.
  • Yalcin, I. (1995). Nonlinear factor analysis. Ph.D. dissertation, Iowa State Univ. Ames, IA.
  • Yalcin, I. and Amemiy a, Y. (1995). Additive nonlinear factor analysis. Preprint 95-16, Statistical Laboratory, Iowa State Univ. Ames, IA.