Electronic Journal of Statistics

Non-Metric Partial Least Squares

Giorgio Russolillo

Full-text: Open access

Abstract

In this paper I review covariance-based Partial Least Squares (PLS) methods, focusing on common features of their respective algorithms and optimization criteria. I then show how these algorithms can be adjusted for use as optimal scaling tools. Three new PLS-type algorithms are proposed for the analysis of one, two or several blocks of variables: the Non-Metric NIPALS, the Non-Metric PLS Regression and the Non-Metric PLS Path Modeling, respectively. These algorithms extend the applicability of PLS methods to data measured on different measurement scales, as well as to variables linked by non-linear relationships.

Article information

Source
Electron. J. Statist. Volume 6 (2012), 1641-1669.

Dates
First available in Project Euclid: 26 September 2012

Permanent link to this document
https://projecteuclid.org/euclid.ejs/1348665231

Digital Object Identifier
doi:10.1214/12-EJS724

Mathematical Reviews number (MathSciNet)
MR2988460

Zentralblatt MATH identifier
1295.62004

Subjects
Primary: 62-07: Data analysis 62H25: Factor analysis and principal components; correspondence analysis

Keywords
Optimal Scaling NIPALS PLS Regression PLS Path Modeling non-linearity

Citation

Russolillo, Giorgio. Non-Metric Partial Least Squares. Electron. J. Statist. 6 (2012), 1641--1669. doi:10.1214/12-EJS724. https://projecteuclid.org/euclid.ejs/1348665231.


Export citation

References

  • [1] Benzécri, J.P. (1973)., L’analyse des Données, Dunod, Paris.
  • [2] Bollen, K.A. (1989)., Structural equations with latent variables, Wiley, New York.
  • [3] Efron, B. (1982)., The Jackknife, the bootstrap, and other resampling plans, SIAM, Philadelphia.
  • [4] Esposito Vinzi, V., Chin W.W., Henseler, J. and Wang, H. (Eds.) (2010)., Handbook of Partial Least Squares (PLS): Concepts, Methods and Applications, Springer Verlag, Berlin Heidelberg.
  • [5] Gifi, A. (1990)., Nonlinear Multivariate Analysis, Wiley, Chichester, UK.
  • [6] Hanafi, M. (2007). PLS Path Modeling: computation of latent variables with estimation mode B, Computational Statistics 22, 275-292.
  • [7] Hayashi, C. (1952). On the prediction of phenomena from qualitative data and the quantification of qualitative data from the mathematico-statistical point of view, Annals of the Institute of Statistical Mathematics 3, 69-98.
  • [8] Höskuldsson, A. (1988). PLS Regression Methods, Journal of Chemometrics 2, 211-228.
  • [9] de Jong, S. (1993). SIMPLS: An alternative approach to partial least squares regression, Chemometrics and Intelligent Laboratory Systems 18, 251-263.
  • [10] Krämer, N. (2007)., Analysis of high-dimensional data with partial least squares and boosting, Phd Thesis,Technischen Universität Berlin, Germany.
  • [11] Kruskal, J. (1964). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis, Psychometrika, 29 (1), 1-27.
  • [12] Kruskal, J. (1964). Nonmetric multidimensional scaling: A numerical method, Psychometrika 29 (2), 115-129.
  • [13] Kruskal, J. and Shepard, R. (1974). A nonmetric variety of linear factor analysis, Psychometrika 39 (2), 123-157.
  • [14] Jakobowicz, E. and Darquenne, C. (2007). A modified PLS path modeling algorithm handling reflective categorical variables and a new model building strategy, Computational Statistics & Data Analysis 51 (8), 3666-3678.
  • [15] Jöreskog, K. G. (1970). A general method for analysis of covariance structures., Biometrika 57, 239-251.
  • [16] de Leeuw, J., Young, F.W. and Takane Y. (1976). Additive structure in qualitative data: an alternating least squares method with optimal scaling features, Psychometrika 41, 471-503.
  • [17] Lohmöller, J.B. (1989)., Latent variable path modeling with partial least squares, Physica-Verlag, Heildelberg.
  • [18] Michailidis, G., de Leeuw, J. (1998). The Gifi system for descriptive multivariate analysis, Statistical Science 13 (4), 307-336.
  • [19] Lyttkens, E. (1966). On the fix-point property of Wold’s iterative estimation method for principal components. In: Krishnaiah P. R. (Ed.), Multivariate Analysis, Academic Press New York, 335-350.
  • [20] R Development Core Team (2011)., R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria. Available at: http://www.r-project.org.
  • [21] Rosipal, R. and Krämer, N. (2006). Overview and Recent Advances in Partial Least Squares. In: C. Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (Eds.), Subspace, Latent Structure and Feature Selection Techniques, Springer, 34-51.
  • [22] Russett, B.M. (1964), Inequality and Instability: the relation of land tenure to politics, World Politics 16, 442-454.
  • [23] Russolillo, G. (2009), PLS methods for Non-Metric data, PhD thesis, Università degli Studi di Napoli “Federico II”, Italy.
  • [24] Russolillo, G. and Lauro N.C. (2011). A Proposal for Handling Categorical Predictors in PLS Regression Framework. In: Fichet, B.; Piccolo, D.; Verde, R.; Vichi, M. (Eds.), Classification and Multivariate Analysis for Complex Data Structures, Springer-Verlag France, 341-348.
  • [25] Russolillo, G., Trinchera L. and Esposito Vinzi, V. (2009). A Non Linear Regularized Component based Approach to Structural Equation Modeling, Book of short papers - Convegno Intermedio SIS-2009 Statistical Methods for the analysis of large data-sets, CLEUP.
  • [26] Sampson, P.D., Streissguth, A.P., Barr, H.M. and Bookstein, F.L. (1989). Neurobehavioral effects of prenatal alcohol: Part II. Partial Least Squares analysis, Neurotoxicology and tetralogy 11 (5), 477-491.
  • [27] Tenenhaus, M. (1977). Analyse en composantes principales d’un ensemble de variables nominales ou numéric, Revue de statistique appliquée 25, 39-56.
  • [28] Tenenhaus, M. (1998)., La Régression PLS: théorie et pratique, Technip, Paris.
  • [29] Tenenhaus, M., Esposito Vinzi, V., Chatelin, Y.-M., Lauro, N.C. (2005). PLS Path Modeling, Computational Statistics & Data Analysis 48, 159-205.
  • [30] Tenenhaus and M, Hanafi, M. (2010), A bridge between PLS path modeling and multi-block data analysis. In: Esposito Vinzi, V., Chin W.W., Henseler, J., Wang, H. (Eds.), Handbook of Partial Least Squares (PLS): Concepts, Methods and Applications, Springer Verlag, Berlin Heidelberg.
  • [31] Tenenhaus, A. and Tenenhaus, M. (2011). Regularized generalized canonical correlation analysis., Psychometrika 76 (2), 257-284.
  • [32] Tishler, A., Dvir, D., Shenhar, A. and Lipovetsky, S. (1996). Identifying critical success factors in defense development projects: a multivariate analysis, Technological forecasting and social change, 51, 151-171.
  • [33] Tucker, L.R. (1958). An Inter-Battery Method of Factor Analysis, Psychometrika 23, 111-136.
  • [34] Wegelin, J.A. (2000)., A survey on Partial Least Squares (PLS) methods, with emphasis on the two-block case, Technical Report N.371, Departement of Statistics, University of Washington, USA: Seattle.
  • [35] Wold, H. (1966). Estimation of principal component and related models by iterative least squares. In: P. R. Krishnaiah (Ed.), Multivariate Analysis, Academic Press, New York, 391-420.
  • [36] Wold, H. (1975). Path Models with Latent Variables: The non-linear iterative partial least squares (NIPALS) approach. In: Blalock, H.M., Aganbegian, A., Borodkin F.M., Boudon, R., Capecchi, V. (Eds.), Quantitative Sociology: Intentional Perspective on Mathematical and Statistical Modeling, Academic Press, New York, 307-357.
  • [37] Wold, H. (1985). Partial Least Squares. In: Kotz, S., Johnson, N.L. (Eds.), Encyclopedia of Statistical Sciences (vol. 6), Wiley, New York, pp. 581-591.
  • [38] Wold, S. (1984), Three PLS algorithms according to SW. In:, Report from the symposium MULTDAST (multivariate analysis in science and methodology), Wold, S. (Ed.), Umeå, Sweden, 26-30.
  • [39] Wold, S., Martens, H. and Wold, H. (1983). The multivariate calibration problem in chemistry solved by the PLS method. In: A. Ruhe, Kagstrom, B. (Eds.), Proceedings of the Conference on Matrix Pencils. Lectures Notes in Mathematics, Springer, Heidelberg.
  • [40] Young, F.W. (1975). Methods for describing ordinal data with cardinal models, Journal of Mathematical Psychology 12, 416-436.
  • [41] Young, F.W. (1981). Quantitative analysis of qualitative data., Psychometrika 44 (4), 357-388.
  • [42] Young, F.W., Takane, Y. and de Leeuw, J. (1978). The principal components of mixed measurement level multivariate data: An alternating least squares method with optimal scaling features, Psychometrika 43 (2), 279-281.