The Annals of Statistics

Penalized maximum likelihood estimation and variable selection in geostatistics

Tingjin Chu, Jun Zhu, and Haonan Wang

Full-text: Open access


We consider the problem of selecting covariates in spatial linear models with Gaussian process errors. Penalized maximum likelihood estimation (PMLE) that enables simultaneous variable selection and parameter estimation is developed and, for ease of computation, PMLE is approximated by one-step sparse estimation (OSE). To further improve computational efficiency, particularly with large sample sizes, we propose penalized maximum covariance-tapered likelihood estimation (PMLET) and its one-step sparse estimation (OSET). General forms of penalty functions with an emphasis on smoothly clipped absolute deviation are used for penalized maximum likelihood. Theoretical properties of PMLE and OSE, as well as their approximations PMLET and OSET using covariance tapering, are derived, including consistency, sparsity, asymptotic normality and the oracle properties. For covariance tapering, a by-product of our theoretical results is consistency and asymptotic normality of maximum covariance-tapered likelihood estimates. Finite-sample properties of the proposed methods are demonstrated in a simulation study and, for illustration, the methods are applied to analyze two real data sets.

Article information

Ann. Statist., Volume 39, Number 5 (2011), 2607-2625.

First available in Project Euclid: 22 December 2011

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62F12: Asymptotic properties of estimators
Secondary: 62M30: Spatial processes

Covariance tapering Gaussian process model selection one-step sparse estimation SCAD spatial linear model


Chu, Tingjin; Zhu, Jun; Wang, Haonan. Penalized maximum likelihood estimation and variable selection in geostatistics. Ann. Statist. 39 (2011), no. 5, 2607--2625. doi:10.1214/11-AOS919.

Export citation


  • [1] Chu, T., Zhu, J. and Wang, H. (2011). Penalized maximum likelihood estiamtion and variable selection in geostatistics. Technical report, Dept. Statistics, Colorado State Univ., Fort Collins, CO.
  • [2] Cressie, N. A. C. (1993). Statistics for Spatial Data, revised ed. Wiley, New York.
  • [3] Draper, N. R. and Smith, H. (1998). Applied Regression Analysis, 3rd ed. Wiley, New York.
  • [4] Du, J., Zhang, H. and Mandrekar, V. S. (2009). Fixed-domain asymptotic properties of tapered maximum likelihood estimators. Ann. Statist. 37 3330–3361.
  • [5] Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression. Ann. Statist. 32 407–499.
  • [6] Fan, J. (1997). Comments on “Wavelets in statistics: A review,” by A. Antoniadis. Journal of the Italian Statistical Association 6 131–138.
  • [7] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360.
  • [8] Fan, J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928–961.
  • [9] Furrer, R., Genton, M. G. and Nychka, D. (2006). Covariance tapering for interpolation of large spatial datasets. J. Comput. Graph. Statist. 15 502–523.
  • [10] Hoeting, J. A., Davis, R. A., Merton, A. A. and Thompson, S. E. (2006). Model selection for geostatistical models. Ecol. Appl. 16 87–98.
  • [11] Huang, H.-C. and Chen, C.-S. (2007). Optimal geostatistical model selection. J. Amer. Statist. Assoc. 102 1009–1024.
  • [12] Kaufman, C. G., Schervish, M. J. and Nychka, D. W. (2008). Covariance tapering for likelihood-based estimation in large spatial data sets. J. Amer. Statist. Assoc. 103 1545–1555.
  • [13] Linhart, H. and Zucchini, W. (1986). Model Selection. Wiley, New York.
  • [14] Mardia, K. V. and Marshall, R. J. (1984). Maximum likelihood estimation of models for residual covariance in spatial regression. Biometrika 71 135–146.
  • [15] Reich, R. and Davis, R. (2008). Lecture Notes of Quantitative Spatial Analysis. Colorado State University, Fort Collins, CO.
  • [16] Schabenberger, O. and Gotway, C. A. (2005). Statistical Methods for Spatial Data Analysis. Chapman and Hall/CRC, Boca Raton, FL.
  • [17] Stein, M. L. (1999). Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York.
  • [18] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267–288.
  • [19] Wang, H., Li, G. and Tsai, C.-L. (2007). Regression coefficient and autoregressive order shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 69 63–78.
  • [20] Wang, H., Li, R. and Tsai, C.-L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94 553–568.
  • [21] Wang, H. and Zhu, J. (2009). Variable selection in spatial regression via penalized least squares. Canad. J. Statist. 37 607–624.
  • [22] Wendland, H. (1995). Piecewise polynomial, positive definite and compactly supported radial functions of minimal degree. Adv. Comput. Math. 4 389–396.
  • [23] Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418–1429.
  • [24] Zou, H. and Li, R. (2008). One-step sparse estimates in nonconcave penalized likelihood models. Ann. Statist. 36 1509–1533.