Open Access
June 2013 Stochastic approximation of score functions for Gaussian processes
Michael L. Stein, Jie Chen, Mihai Anitescu
Ann. Appl. Stat. 7(2): 1162-1191 (June 2013). DOI: 10.1214/13-AOAS627


We discuss the statistical properties of a recently introduced unbiased stochastic approximation to the score equations for maximum likelihood calculation for Gaussian processes. Under certain conditions, including bounded condition number of the covariance matrix, the approach achieves $O(n)$ storage and nearly $O(n)$ computational effort per optimization step, where $n$ is the number of data sites. Here, we prove that if the condition number of the covariance matrix is bounded, then the approximate score equations are nearly optimal in a well-defined sense. Therefore, not only is the approximation efficient to compute, but it also has comparable statistical properties to the exact maximum likelihood estimates. We discuss a modification of the stochastic approximation in which design elements of the stochastic terms mimic patterns from a $2^{n}$ factorial design. We prove these designs are always at least as good as the unstructured design, and we demonstrate through simulation that they can produce a substantial improvement over random designs. Our findings are validated by numerical experiments on simulated data sets of up to 1 million observations. We apply the approach to fit a space–time model to over 80,000 observations of total column ozone contained in the latitude band $40^{\circ}\mathrm{-}50^{\circ}\mathrm{N}$ during April 2012.


Download Citation

Michael L. Stein. Jie Chen. Mihai Anitescu. "Stochastic approximation of score functions for Gaussian processes." Ann. Appl. Stat. 7 (2) 1162 - 1191, June 2013.


Published: June 2013
First available in Project Euclid: 27 June 2013

zbMATH: 06279869
MathSciNet: MR3113505
Digital Object Identifier: 10.1214/13-AOAS627

Keywords: Gaussian process , Hutchinson trace estimators , iterative methods , maximum likelihood , preconditioning , unbiased estimating equations

Rights: Copyright © 2013 Institute of Mathematical Statistics

Vol.7 • No. 2 • June 2013
Back to Top