Translator Disclaimer
October 2021 Variable selection consistency of Gaussian process regression
Sheng Jiang, Surya T. Tokdar
Author Affiliations +
Ann. Statist. 49(5): 2491-2505 (October 2021). DOI: 10.1214/20-AOS2043

Abstract

Bayesian nonparametric regression under a rescaled Gaussian process prior offers smoothness-adaptive function estimation with near minimax-optimal error rates. Hierarchical extensions of this approach, equipped with stochastic variable selection, are known to also adapt to the unknown intrinsic dimension of a sparse true regression function. But it remains unclear if such extensions offer variable selection consistency, that is, if the true subset of important variables could be consistently learned from the data. It is shown here that variable consistency may indeed be achieved with such models at least when the true regression function has finite smoothness to induce a polynomially larger penalty on inclusion of false positive predictors. Our result covers the high-dimensional asymptotic setting where the predictor dimension is allowed to grow with the sample size. The proof utilizes Schwartz theory to establish that the posterior probability of wrong selection vanishes asymptotically. A necessary and challenging technical development involves providing sharp upper and lower bounds to small ball probabilities at all rescaling levels of the Gaussian process prior, a result that could be of independent interest.

Citation

Download Citation

Sheng Jiang. Surya T. Tokdar. "Variable selection consistency of Gaussian process regression." Ann. Statist. 49 (5) 2491 - 2505, October 2021. https://doi.org/10.1214/20-AOS2043

Information

Received: 1 December 2019; Revised: 1 November 2020; Published: October 2021
First available in Project Euclid: 12 November 2021

Digital Object Identifier: 10.1214/20-AOS2043

Subjects:
Primary: 62G08 , 62G20
Secondary: 60G05

Keywords: adaptive estimation , Bayesian inference , Gaussian process priors , high-dimensional regression , nonparametric variable selection

Rights: Copyright © 2021 Institute of Mathematical Statistics

JOURNAL ARTICLE
15 PAGES

This article is only available to subscribers.
It is not available for individual sale.
+ SAVE TO MY LIBRARY

SHARE
Vol.49 • No. 5 • October 2021
Back to Top