Electronic Journal of Statistics
- Electron. J. Statist.
- Volume 5 (2011), 127-145.
PAC-Bayesian bounds for sparse regression estimation with exponential weights
We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering high-dimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The Lasso is solution of a convex minimization problem, hence computable for large value of p. However stringent conditions on the design are required to establish fast rates of convergence for this estimator. Dalalyan and Tsybakov [17–19] proposed an exponential weights procedure achieving a good compromise between the statistical and computational aspects. This estimator can be computed for reasonably large p and satisfies a sparsity oracle inequality in expectation for the empirical excess risk only under mild assumptions on the design. In this paper, we propose an exponential weights estimator similar to that of  but with improved statistical performances. Our main result is a sparsity oracle inequality in probability for the true excess risk.
Electron. J. Statist. Volume 5 (2011), 127-145.
First available in Project Euclid: 14 March 2011
Permanent link to this document
Digital Object Identifier
Mathematical Reviews number (MathSciNet)
Zentralblatt MATH identifier
Primary: 62J07: Ridge regression; shrinkage estimators
Secondary: 62J05: Linear regression 62G08: Nonparametric regression 62F15: Bayesian inference 62B10: Information-theoretic topics [See also 94A17] 68T05: Learning and adaptive systems [See also 68Q32, 91E40]
Alquier, Pierre; Lounici, Karim. PAC-Bayesian bounds for sparse regression estimation with exponential weights. Electron. J. Statist. 5 (2011), 127--145. doi:10.1214/11-EJS601. https://projecteuclid.org/euclid.ejs/1300108317