Electronic Journal of Statistics

Maximum likelihood estimation for Gaussian processes under inequality constraints

François Bachoc, Agnès Lagnoux, and Andrés F. López-Lopera

Full-text: Open access


We consider covariance parameter estimation for a Gaussian process under inequality constraints (boundedness, monotonicity or convexity) in fixed-domain asymptotics. We address the estimation of the variance parameter and the estimation of the microergodic parameter of the Matérn and Wendland covariance functions. First, we show that the (unconstrained) maximum likelihood estimator has the same asymptotic distribution, unconditionally and conditionally to the fact that the Gaussian process satisfies the inequality constraints. Then, we study the recently suggested constrained maximum likelihood estimator. We show that it has the same asymptotic distribution as the (unconstrained) maximum likelihood estimator. In addition, we show in simulations that the constrained maximum likelihood estimator is generally more accurate on finite samples. Finally, we provide extensions to prediction and to noisy observations.

Article information

Electron. J. Statist., Volume 13, Number 2 (2019), 2921-2969.

Received: September 2018
First available in Project Euclid: 3 September 2019

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62M30: Spatial processes
Secondary: 62F12: Asymptotic properties of estimators

Gaussian processes inequality constraints fixed-domain asymptotics constrained maximum likelihood asymptotic normality

Creative Commons Attribution 4.0 International License.


Bachoc, François; Lagnoux, Agnès; López-Lopera, Andrés F. Maximum likelihood estimation for Gaussian processes under inequality constraints. Electron. J. Statist. 13 (2019), no. 2, 2921--2969. doi:10.1214/19-EJS1587. https://projecteuclid.org/euclid.ejs/1567497627

Export citation


  • [1] P. Abrahamsen. A review of Gaussian random fields and correlation functions. Technical report, Norwegian computing center, 1997.
  • [2] M. Abramowitz and I. A. Stegun., Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York, ninth Dover printing, tenth GPO printing edition, 1964.
  • [3] R. J. Adler., An introduction to continuity, extrema, and related topics for general Gaussian processes. IMS, 1990.
  • [4] R. J. Adler and J. E. Taylor., Random fields and geometry. Springer Monographs in Mathematics. Springer, New York, 2007.
  • [5] E. Anderes. On the consistent separation of scale and variance for Gaussian random fields., The Annals of Statistics, 38:870–893, 2010.
  • [6] J.-M. Azaïs, F. Bachoc, T. Klein, A. Lagnoux, and T. M. N. Nguyen. Semi-parametric estimation of the variogram of a Gaussian process with stationary increments., arXiv :1806.03135, 2018.
  • [7] J.-M. Azaïs and M. Wschebor., Level sets and extrema of random processes and fields. John Wiley & Sons, Inc., Hoboken, NJ, 2009.
  • [8] F. Bachoc. Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model mispecification., Computational Statistics and Data Analysis, 66:55–69, 2013.
  • [9] F. Bachoc. Asymptotic analysis of the role of spatial sampling for covariance parameter estimation of Gaussian processes., Journal of Multivariate Analysis, 125:1–35, 2014.
  • [10] F. Bachoc. Asymptotic analysis of the role of spatial sampling for covariance parameter estimation of Gaussian processes., Journal of Multivariate Analysis, 125:1–35, 2014.
  • [11] F. Bachoc, K. Ammar, and J. Martinez. Improvement of code behavior in a design of experiments by metamodeling., Nuclear science and engineering, 183(3):387–406, 2016.
  • [12] F. Bachoc, G. Bois, J. Garnier, and J. Martinez. Calibration and improved prediction of computer models by universal Kriging., Nuclear Science and Engineering, 176(1):81–97, 2014.
  • [13] F. Bachoc et al. Asymptotic analysis of covariance parameter estimation for Gaussian processes in the misspecified case., Bernoulli, 24(2) :1531–1575, 2018.
  • [14] J. Bect, F. Bachoc, and D. Ginsbourger. A supermartingale approach to Gaussian process based sequential design of experiments., Bernoulli, forthcoming, 2018.
  • [15] M. Bevilacqua, T. Faouzi, R. Furrer, and E. Porcu. Estimation and prediction using generalized Wendland covariance functions under fixed domain asymptotics., The Annals of Statistics, 47(2):828–856, 2019.
  • [16] C.-H. Chang, H.-C. Huang, C.-K. Ing, et al. Mixed domain asymptotics for a stochastic process model with time trend and measurement error., Bernoulli, 23(1):159–190, 2017.
  • [17] H.-S. Chen, D. Simpson, and Z. Ying. Infill asymptotics for a stochastic process model with measurement error., Statistica Sinica, 10:141–156, 2000.
  • [18] A. Cousin, H. Maatouk, and D. Rullière. Kriging of financial term-structures., European Journal of Operational Research, 255(2):631–648, 2016.
  • [19] S. Da Veiga and A. Marrel. Gaussian process modeling with inequality constraints., Annales de la faculté des sciences de Toulouse Mathématiques, 21(3):529–555, 4 2012.
  • [20] J. Du, H. Zhang, and V. Mandrekar. Fixed-domain asymptotic properties of tapered maximum likelihood estimators., The Annals of Statistics, 37 :3330–3361, 2009.
  • [21] T. Gneiting. Compactly supported correlation functions., Journal of Multivariate Analysis, 83(2):493–508, 2002.
  • [22] S. Golchi, D. R. Bingham, H. Chipman, and D. A. Campbell. Monotone emulation of computer experiments., SIAM/ASA Journal on Uncertainty Quantification, 3(1):370–392, 2015.
  • [23] I. Ibragimov and Y. Rozanov., Gaussian Random Processes. Springer-Verlag, New York, 1978.
  • [24] J. Istas and G. Lang. Quadratic variations and estimation of the local Hölder index of a Gaussian process., Annales de l’Institut Henri Poincaré, 33:407–436, 1997.
  • [25] D. Jones, M. Schonlau, and W. Welch. Efficient global optimization of expensive black box functions., Journal of Global Optimization, 13:455–492, 1998.
  • [26] C. Kaufman and B. Shaby. The role of the range parameter for estimation and prediction in geostatistics., Biometrika, 100:473–484, 2013.
  • [27] M. A. Lifshits., Gaussian random functions, volume 322 of Mathematics and its Applications. Kluwer Academic Publishers, Dordrecht, 1995.
  • [28] A. F. López-Lopera., lineqGPR: Gaussian Process Regression Models with Linear Inequality Constraints, 2018. R package version 0.0.3.
  • [29] A. F. López-Lopera, F. Bachoc, N. Durrande, and O. Roustand. Finite-dimensional Gaussian approximation with linear inequality constraints., SIAM/ASA Journal on Uncertainty Quantification, 6(3) :1224–1255, 2018.
  • [30] A. F. López-lopera, S. John, and N. Durrande. Gaussian process modulated Cox processes under linear inequality constraints. In, AISTATS, volume 89 of Proceedings of Machine Learning Research, pages 1997–2006, 16–18 Apr 2019.
  • [31] H. Maatouk and X. Bay. Gaussian process emulators for computer experiments with inequality constraints., Mathematical Geosciences, 49(5):557–582, 2017.
  • [32] G. Matheron., La Théorie des Variables Régionalisées et ses Applications. Fasicule 5 in Les Cahiers du Centre de Morphologie Mathématique de Fontainebleau. Ecole Nationale Supérieure des Mines de Paris, 1970.
  • [33] H. Nickisch and C. E. Rasmussen. Approximations for binary Gaussian process classification., Journal of Machine Learning Research, 9(Oct) :2035–2078, 2008.
  • [34] A. Pakman and L. Paninski. Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians., Journal of Computational and Graphical Statistics, 23(2):518–542, 2014.
  • [35] R. Paulo, G. Garcia-Donato, and J. Palomo. Calibration of computer models with multivariate output., Computational Statistics and Data Analysis, 56 :3959–3974, 2012.
  • [36] H. Putter and G. A. Young. On the effect of covariance function estimation on the accuracy of Kriging predictors., Bernoulli, 7(3):421–438, 2001.
  • [37] C. Rasmussen and C. Williams., Gaussian Processes for Machine Learning. The MIT Press, Cambridge, 2006.
  • [38] J. Riihimäki and A. Vehtari. Gaussian processes with monotonicity information. In, Journal of Machine Learning Research: Workshop and Conference Proceedings, volume 9, pages 645–652, 2010.
  • [39] O. Roustant, D. Ginsbourger, Y. Deville, et al. DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by Kriging-based metamodeling and optimization., Journal of Statistical Software, 51(i01), 2012.
  • [40] J. Sacks, W. Welch, T. Mitchell, and H. Wynn. Design and analysis of computer experiments., Statistical Science, 4:409–423, 1989.
  • [41] T. Santner, B. Williams, and W. Notz., The Design and Analysis of Computer Experiments. Springer, New York, 2003.
  • [42] M. Stein. Asymptotically efficient prediction of a random field with a misspecified covariance function., The Annals of Statistics, 16:55–63, 1988.
  • [43] M. Stein. Bounds on the efficiency of linear predictions using an incorrect covariance function., The Annals of Statistics, 18 :1116–1138, 1990.
  • [44] M. Stein. A comparison of generalized cross validation and modified maximum likelihood for estimating the parameters of a stochastic process., The Annals of Statistics, 18 :1139–1157, 1990.
  • [45] M. Stein. Uniform asymptotic optimality of linear predictions of a random field using an incorrect second-order structure., The Annals of Statistics, 18:850–872, 1990.
  • [46] M. Stein., Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York, 1999.
  • [47] A. W. van der Vaart., Asymptotic statistics, volume 3 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998.
  • [48] D. Wang., Fixed Domain Asymptotics and Consistent Estimation for Gaussian Random Field Models in Spatial Statistics and Computer Experiments. PhD thesis, Nat. Univ. Singapore, Singapore, 2010.
  • [49] D. Wang and W.-L. Loh. On fixed-domain asymptotics and covariance tapering in Gaussian random field models., Electronic Journal of Statistics, 5:238–269, 2011.
  • [50] H. Wendland., Scattered data approximation, volume 17. Cambridge university press, 2004.
  • [51] Z. Ying. Asymptotic properties of a maximum likelihood estimator with data from a Gaussian process., Journal of Multivariate Analysis, 36:280–296, 1991.
  • [52] Z. Ying. Maximum likelihood estimation of parameters under a spatial sampling scheme., The Annals of Statistics, 21 :1567–1590, 1993.
  • [53] H. Zhang. Inconsistent estimation and asymptotically equivalent interpolations in model-based geostatistics., Journal of the American Statistical Association, 99:250–261, 2004.
  • [54] H. Zhang and Y. Wang. Kriging and cross validation for massive spatial data., Environmetrics, 21:290–304, 2010.
  • [55] D.-X. Zhou. Derivative reproducing properties for kernel methods in learning theory., Journal of computational and Applied Mathematics, 220(1):456–463, 2008.