Annals of Applied Statistics

Nonseparable dynamic nearest neighbor Gaussian process models for large spatio-temporal data with an application to particulate matter analysis

Abhirup Datta, Sudipto Banerjee, Andrew O. Finley, Nicholas A. S. Hamm, and Martijn Schaap

Full-text: Open access


Particulate matter (PM) is a class of malicious environmental pollutants known to be detrimental to human health. Regulatory efforts aimed at curbing PM levels in different countries often require high resolution space–time maps that can identify red-flag regions exceeding statutory concentration limits. Continuous spatio-temporal Gaussian Process (GP) models can deliver maps depicting predicted PM levels and quantify predictive uncertainty. However, GP-based approaches are usually thwarted by computational challenges posed by large datasets. We construct a novel class of scalable Dynamic Nearest Neighbor Gaussian Process (DNNGP) models that can provide a sparse approximation to any spatio-temporal GP (e.g., with nonseparable covariance structures). The DNNGP we develop here can be used as a sparsity-inducing prior for spatio-temporal random effects in any Bayesian hierarchical model to deliver full posterior inference. Storage and memory requirements for a DNNGP model are linear in the size of the dataset, thereby delivering massive scalability without sacrificing inferential richness. Extensive numerical studies reveal that the DNNGP provides substantially superior approximations to the underlying process than low-rank approximations. Finally, we use the DNNGP to analyze a massive air quality dataset to substantially improve predictions of PM levels across Europe in conjunction with the LOTOS-EUROS chemistry transport models (CTMs).

Article information

Ann. Appl. Stat., Volume 10, Number 3 (2016), 1286-1316.

Received: September 2015
Revised: March 2016
First available in Project Euclid: 28 September 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Nonseparable spatio-temporal models scalable Gaussian process nearest neighbors Bayesian inference Markov chain Monte Carlo environmental pollutants


Datta, Abhirup; Banerjee, Sudipto; Finley, Andrew O.; Hamm, Nicholas A. S.; Schaap, Martijn. Nonseparable dynamic nearest neighbor Gaussian process models for large spatio-temporal data with an application to particulate matter analysis. Ann. Appl. Stat. 10 (2016), no. 3, 1286--1316. doi:10.1214/16-AOAS931.

Export citation


  • Allcroft, D. J. and Glasbey, C. A. (2003). A latent Gaussian Markov random-field model for spatiotemporal rainfall disaggregation. J. Roy. Statist. Soc. Ser. C 52 487–498.
  • Bai, Y., Song, P. X. K. and Raghunathan, T. E. (2012). Bayesian dynamic modeling for large space–time datasets using Gaussian predictive processes. J. Roy. Statist. Soc. Ser. B 74 799–824.
  • Banerjee, S., Carlin, B. P. and Gelfand, A. E. (2014). Hierarchical Modeling and Analysis for Spatial Data, 2nd ed. Chapman & Hall, Boca Raton, FL.
  • Banerjee, S., Gelfand, A. E., Finley, A. O. and Sang, H. (2008). Gaussian predictive process models for large spatial data sets. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 825–848.
  • Bevilacqua, M., Gaetan, C., Mateu, J. and Porcu, E. (2012). Estimating space and space–time covariance functions for large data sets: A weighted composite likelihood approach. J. Amer. Statist. Assoc. 107 268–280.
  • Bevilacqua, M., Fass, ò. A., Gaetan, C., Porcu, E.and Velandia, D. (2015). Covariance tapering for multivariate Gaussian random fields estimation. Stat. Methods Appl. 25 21–37.
  • Birmili, W., Schepanski, K., Ansmann, A., Spindler, G., Tegen, I., Wehner, B., Nowak, A., Reimer, E., Mattis, I., Muller, K., Bruggemann, E., Gnauk, T., Herrmann, H., Wiedensohler, A., Althausen, D., Schladitz, A., Tuch, T. and Loschau, G. (2008). A case of extreme particulate matter concentrations over central Europe caused by dust emitted over the southern Ukraine. Atmos. Chem. Phys. 8 997–1016.
  • Brauer, M., Amann, M., Burnett, R. T., Cohen, A., Dentener, F., Ezzati, M., Henderson, S. B., Krzyzanowski, M., Martin, R. V., Van Dingenen, R., van Donkelaar, A. and Thurston, G. D. (2011). Exposure assessment for estimation of the global burden of disease attributable to outdoor air pollution. Environ. Sci. Technol. 46 652–660.
  • Brunekreef, B. and Holgate, S. T. (2002). Air pollution and health. Lancet 360 1233–1242.
  • Candiani, G., Carnevale, C., Finzi, G., Pisoni, E. and Volta, M. (2013). A comparison of reanalysis techniques: Applying optimal interpolation and ensemble Kalman filtering to improve air quality monitoring at mesoscale. Sci. Total Environ. 458–460 7–14.
  • Crainiceanu, C. M., Diggle, P. J. and Rowlingson, B. (2008). Bivariate binomial spatial modeling of Loa loa prevalence in tropical Africa. J. Amer. Statist. Assoc. 103 21–37.
  • Cressie, N. and Huang, H.-C. (1999). Classes of nonseparable, spatio-temporal stationary covariance functions. J. Amer. Statist. Assoc. 94 1330–1340.
  • Cressie, N. and Johannesson, G. (2008). Fixed rank kriging for very large spatial data sets. J. R. Stat. Soc. Ser. B Stat. Methodol. 70 209–226.
  • Cressie, N., Shi, T. and Kang, E. L. (2010). Fixed rank filtering for spatio-temporal data. J. Comput. Graph. Statist. 19 724–745.
  • Cressie, N. and Wikle, C. K. (2011). Statistics for Spatio-Temporal Data. Wiley, Hoboken, NJ.
  • Dagum, L. and Menon, R. (1998). OpenMP: An industry standard API for shared-memory programming. IEEE Comput. Sci. Eng. 5 46–55.
  • Datta, A., Banerjee, S., Finley, A. O. and Gelfand, A. E. (2016a). Hierarchical nearest-neighbor Gaussian process models for large geostatistical datasets. J. Amer. Statist. Assoc. 111 800–812.
  • Datta, A., Banerjee, S., Finley, A. O., Hamm, N. S. and Schaap, M. (2016b). Supplement to “Nonseparable dynamic nearest neighbor Gaussian process models for large spatio-temporal data with an application to particulate matter analysis.” DOI:10.1214/16-AOAS931SUPP.
  • Denby, B., Schaap, M., Segers, A., Builtjes, P. and Horalek, J. (2008). Comparison of two data assimilation methods for assessing PM10 exceedances on the European scale. Atmos. Environ. 42 7122–7134.
  • Denby, B., Sundvor, I., Cassiani, M., de Smet, P., de Leeuw, F. and Horalek, J. (2010). Spatial mapping of ozone and SO2 trends in Europe. Sci. Total Environ. 408 4795–4806.
  • Du, J., Zhang, H. and Mandrekar, V. S. (2009). Fixed-domain asymptotic properties of tapered maximum likelihood estimators. Ann. Statist. 37 3330–3361.
  • Eeftens, M., Tsai, M. Y., Ampe, C., Anwander, B., Beelen, R., Bellander, T., Cesaroni, G., Cirach, M., Cyrys, J., de Hoogh, K., De Nazelle, A., de Vocht, F., Declercq, C., Dedele, A., Eriksen, K., Galassi, C., Grazuleviciene, R., Grivas, G., Heinrich, J., Hoffmann, B., Iakovides, M., Ineichen, A., Katsouyanni, K., Korek, M., Kramer, U., Kuhlbusch, T., Lanki, T., Madsen, C., Meliefste, K., Molter, A., Mosler, G., Nieuwenhuijsen, M., Oldenwening, M., Pennanen, A., Probst-Hensch, N., Quass, U., Raaschou-Nielsen, O., Ranzi, A., Stephanou, E., Sugiri, D., Udvardy, O., Vaskoevi, E., Weinmayr, G., Brunekreef, B. and Hoek, G. (2012). Spatial variation of PM2.5, PM10, PM2.5 absorbance and PMcoarse concentrations between and within 20 European study areas and the relationship with NO2—results of the ESCAPE project. Atmos. Environ. 62 303–317.
  • Eidsvik, J., Shaby, B. A., Reich, B. J., Wheeler, M. and Niemi, J. (2014). Estimation and prediction in spatial models with block composite likelihoods. J. Comput. Graph. Statist. 23 295–315.
  • European Commission (2015). European Union Air Quality Standards. Available at
  • Finley, A. O., Banerjee, S. and Gelfand, A. E. (2012). Bayesian dynamic modeling for large space–time datasets using Gaussian predictive processes. J. Geogr. Syst. 14 29–47.
  • Finley, A. O., Banerjee, S. and McRoberts, R. E. (2009). Hierarchical spatial models for predicting tree species assemblages across large domains. Ann. Appl. Stat. 3 1052–1079.
  • Flemming, J., Inness, A., Flentje, H., Huijnen, V., Moinat, P., Schultz, M. G. and Stein, O. (2009). Coupling global chemistry transport models to ECMWF’s integrated forecast system. Geosci. Model Dev. 2 253–265.
  • Furrer, R., Genton, M. G. and Nychka, D. (2006). Covariance tapering for interpolation of large spatial datasets. J. Comput. Graph. Statist. 15 502–523.
  • Gelfand, A. E., Banerjee, S. and Gamerman, D. (2005). Spatial process modelling for univariate and multivariate dynamic spatial data. Environmetrics 16 465–479.
  • Gelfand, A. E. and Ghosh, S. K. (1998). Model choice: A minimum posterior predictive loss approach. Biometrika 85 1–11.
  • Gelfand, A. E., Diggle, P. J., Fuentes, M. and Guttorp, P., eds. (2010). Handbook of Spatial Statistics. Chapman & Hall/CRC Handbooks of Modern Statistical Methods. CRC Press, Boca Raton, FL.
  • Gneiting, T. (2002). Nonseparable, stationary covariance functions for space–time data. J. Amer. Statist. Assoc. 97 590–600.
  • Gneiting, T., Genton, M. G. and Guttorp, P. (2007). Geostatistical space–time models, stationarity, separability and full symmetry. In Statistics of SpatioTemporal Systems (B. Finkenstaedt, L. Held andV. Isham, eds.) 151–175. Chapman & Hall, London.
  • Gneiting, T. and Guttorp, P. (2010). Continuous parameter spatio-temporal processes. In Handbook of Spatial Statistics. Handb. Mod. Stat. Methods (A. E. Gelfand, P. Diggle, M. Fuentes and P. Guttorp, eds.) 427–436. CRC Press, Boca Raton, FL.
  • Gräler, B., Gerharz, L. and Pebesma, E. (2011). Spatio-temporal analysis and interpolation of PM10 measurements in Europe. ETC/ACM Technical Paper 2011/10, European Topic Centre on Air Pollution and Climate Change Mitigation, Bilthoven, The Netherlands.
  • Gramacy, R. B. and Apley, D. W. (2015). Local Gaussian process approximation for large computer experiments. J. Comput. Graph. Statist. 24 561–578.
  • Hamm, N. A. S., Finley, A. O., Schaap, M. and Stein, A. (2015). A spatially varying coefficient model for mapping PM10 air quality at the European scale. Atmos. Environ. 102 393–405.
  • Hendriks, C., Kranenburg, R., Kuenen, J., van Gijlswijk, R., Kruit, R. W., Segers, A., van der Gon, H. D. and Schaap, M. (2013). The origin of ambient particulate matter concentrations in the Netherlands. Atmospheric Environment 69 289–303.
  • Higdon, D. (2001). Space and space time modeling using process convolutions. Technical report, Institute of Statistics and Decision Sciences, Duke Univ., Durham, NC.
  • Hoek, G., Krishnan, R. M., Beelen, R., Peters, A., Ostro, B., Brunekreef, B. and Kaufman, J. D. (2013). Long-term air pollution exposure and cardio-respiratory mortality: A review. Environ. Health 12 43.
  • Intel (2015). Math Kernel Library. Available at
  • Jones, R. H. and Zhang, Y. (1997). Models for continuous stationary space–time processes. In Modelling Longitudinal and Spatially Correlated Data (T. G. Gregoire, D. R. Brillinger, P. J. Diggle, E. Russek-Cohen, W. G. Warren andR. D. Wolfinger, eds.) 289–298. Springer, New York.
  • Kammann, E. E. and Wand, M. P. (2003). Geoadditive models. J. Roy. Statist. Soc. Ser. C 52 1–18.
  • Katzfuss, M. (2016). A multi-resolution approximation for massive spatial datasets. J. Amer. Statist. Assoc. Available at arXiv:1507.04789.
  • Katzfuss, M. and Cressie, N. (2012). Bayesian hierarchical spatio-temporal smoothing for very large datasets. Environmetrics 23 94–107.
  • Kaufman, C. G., Schervish, M. J. and Nychka, D. W. (2008). Covariance tapering for likelihood-based estimation in large spatial data sets. J. Amer. Statist. Assoc. 103 1545–1555.
  • Kyriakidis, P. C. and Journel, A. G. (1999). Geostatistical space–time models: A review. Math. Geol. 31 651–684.
  • Lloyd, C. D. and Atkinson, P. M. (2004). Increased accuracy of geostatistical prediction of nitrogen dioxide in the United Kingdom with secondary data. International Journal of Applied Earth Observation and Geoinformation 5 293–305.
  • Loomis, D., Grosse, Y., Lauby-Secretan, B., El Ghissassi, F., Bouvard, V., Benbrahim-Tallaa, L., Guha, N., Baan, R., Mattock, H. and Straif, S. (2013). The carcinogenicity of outdoor air pollution. Lancet Oncol. 14 1262–1263.
  • Manders, A. M. M., Schaap, M. and Hoogerbrugge, R. (2009). Testing the capability of the chemistry transport model LOTOS-EUROS to forecast PM10 levels in the Netherlands. Atmos. Environ. 43 4050–4059.
  • Mues, A., Kuenen, J., Hendriks, C., Manders, A., Segers, A., Scholz, Y., Hueglin, C., Builtjes, P. and Schaap, M. (2014). Sensitivity of air pollution simulations with LOTOS-EUROS to the temporal distribution of anthropogenic emissions. Atmos. Chem. Phys. 14 939–955.
  • Omidi, M. and Mohammadzadeh, M. (2015). A new method to build spatio-temporal covariance functions: Analysis of ozone data. Statist. Papers 1–15.
  • Pfeifer, P. E. and Deutsch, S. J. (1980a). Independence and sphericity tests for the residuals of space–time ARMA models. Comm. Statist. Simulation Comput. 9 533–549.
  • Pfeifer, P. E. and Deutsch, S. J. (1980b). Stationarity and invertibility regions for low order STARMA models. Comm. Statist. Simulation Comput. 9 551–562.
  • Pouliot, G., Pierce, T., van der Gon, H. D., Schaap, M., Moran, M. and Nopmongcol, U. (2012). Comparing emission inventories and model-ready emission datasets between Europe and North America for the AQMEII project. Atmos. Environ. 53 4–14.
  • R’Honi, Y., Clarisse, L., Clerbaux, C., Hurtmans, D., Duflot, V., Turquety, S., Ngadi, Y. and Coheur, P. F. (2013). Exceptional emissions of NH3 and HCOOH in the 2010 Russian wildfires. Atmos. Chem. Phys. 13 4171–4181.
  • Rasmussen, C. E. and Williams, C. K. I. (2005). Gaussian Processes for Machine Learning, 1st ed. MIT Press, Cambridge, MA.
  • Rue, H. and Held, L. (2005). Gaussian Markov Random Fields: Theory and Applications. Monographs on Statistics and Applied Probability 104. Chapman & Hall, Boca Raton, FL.
  • Sang, H. and Huang, J. Z. (2012). A full scale approximation of covariance functions for large spatial data sets. J. R. Stat. Soc. Ser. B. Stat. Methodol. 74 111–132.
  • Schaap, M., Timmermans, R. M. A., Roemer, M., Boersen, G. A. C., Builtjes, P., Sauter, F., Velders, G. and Beck, J. (2008). The LOTOS-EUROS model: Description, validation and latest developments. Int. J. Environ. Pollut. 32 270–290.
  • Shaby, B. and Ruppert, D. (2012). Tapered covariance: Bayesian estimation and asymptotics. J. Comput. Graph. Statist. 21 433–452.
  • Spiegelhalter, D. J., Best, N. G., Carlin, B. P. and van der Linde, A. (2002). Bayesian measures of model complexity and fit. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 583–639.
  • Stein, M. L. (2005). Space–time covariance functions. J. Amer. Statist. Assoc. 100 310–321.
  • Stein, M. L. (2007). Spatial variation of total column ozone on a global scale. Ann. Appl. Stat. 1 191–210.
  • Stein, M. L. (2008). A modeling approach for large spatial datasets. J. Korean Statist. Soc. 37 3–10.
  • Stein, M. L. (2013). On a class of space–time intrinsic random functions. Bernoulli 19 387–408.
  • Stein, M. L. (2014). Limitations on low rank approximations for covariance matrices of spatial data. Spat. Stat. 8 1–19.
  • Stein, M. L., Chi, Z. and Welty, L. J. (2004). Approximating likelihoods for large spatial data sets. J. R. Stat. Soc. Ser. B Stat. Methodol. 66 275–296.
  • Stern, R., Builtjes, P., Schaap, M., Timmermans, R., Vautard, R., Hodzic, A., Memmesheimer, M., Feldmann, H., Renner, E., Wolke, R. and Kerschbaumer, A. (2008). A model inter-comparison study focussing on episodes with elevated PM10 concentrations. Atmos. Environ. 42 4567–4588.
  • Stoffer, D. S. (1986). Estimation and identification of space–time ARMAX models in the presence of missing data. J. Amer. Statist. Assoc. 81 762–772.
  • Stroud, J. R., Müller, P. and Sansó, B. (2001). Dynamic models for spatiotemporal data. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 673–689.
  • van de Kassteele, J. and Stein, A. (2006). A model for external drift kriging with uncertain covariates applied to air quality measurements and dispersion model output. Environmetrics 17 309–322.
  • Vecchia, A. V. (1988). Estimation and model identification for continuous spatial processes. J. Roy. Statist. Soc. Ser. B 50 297–312.
  • Vecchia, A. V. (1992). A new method of prediction for spatial regression models with correlated errors. J. Roy. Statist. Soc. Ser. B 54 813–830.
  • Xu, G., Liang, F. and Genton, M. G. (2015). A Bayesian spatio-temporal geostatistical model with an auxiliary lattice for large datasets. Statist. Sinica. 25 61–79.
  • Yeniay, Ö. and Göktaş, A. (2002). A comparison of partial least squares regression with other prediction methods. Hacet. J. Math. Stat. 31 99–111.

Supplemental materials

  • Supplement to “Nonseparable dynamic nearest neighbor Gaussian process models for large spatio-temporal data with an application to particulate matter analysis”. File containing supplementary materials including a formal construction of eligible sets, additional simulation experiments and possible extension of DNNGP to model nonstationary covariances.