The Annals of Statistics

Properties and refinements of the fused lasso

Alessandro Rinaldo

Full-text: Open access

Abstract

We consider estimating an unknown signal, both blocky and sparse, which is corrupted by additive noise. We study three interrelated least squares procedures and their asymptotic properties. The first procedure is the fused lasso, put forward by Friedman et al. [Ann. Appl. Statist. 1 (2007) 302–332], which we modify into a different estimator, called the fused adaptive lasso, with better properties. The other two estimators we discuss solve least squares problems on sieves; one constrains the maximal 1 norm and the maximal total variation seminorm, and the other restricts the number of blocks and the number of nonzero coordinates of the signal. We derive conditions for the recovery of the true block partition and the true sparsity patterns by the fused lasso and the fused adaptive lasso, and we derive convergence rates for the sieve estimators, explicitly in terms of the constraining parameters.

Article information

Source
Ann. Statist., Volume 37, Number 5B (2009), 2922-2952.

Dates
First available in Project Euclid: 17 July 2009

Permanent link to this document
https://projecteuclid.org/euclid.aos/1247836673

Digital Object Identifier
doi:10.1214/08-AOS665

Mathematical Reviews number (MathSciNet)
MR2541451

Zentralblatt MATH identifier
1173.62027

Subjects
Primary: 62G08: Nonparametric regression 62G20: Asymptotic properties

Keywords
Fused lasso consistency sieve least squares

Citation

Rinaldo, Alessandro. Properties and refinements of the fused lasso. Ann. Statist. 37 (2009), no. 5B, 2922--2952. doi:10.1214/08-AOS665. https://projecteuclid.org/euclid.aos/1247836673


Export citation

References

  • Birman, M. S. and Solomjak, M. Z. (1967). Piece-wise polynomial approximations of functions in the class Wpα. Mat. Sb. (N.S.) 73 295–317.
  • Boysen, L., Kempe, A., Liebscher, V., Munk, A. and Wittich, Ol. (2009). Consistencies and rates of convergence of jump-penalized least squares estimators. Ann. Statist. 37 157–183.
  • Caselles, V., Chambolle, A. and Novaga, M. (2007). The discontinuity set of solutions of the TV denoising problem and some extensions. Multiscale Model. Simul. 6 879–894.
  • Davies, P. L. and Kovac, A. (2001a). Local extremes, runs, strings and multiresolution. Ann. Statist. 29 1–48. With discussion and rejoinder by the authors.
  • Davies, P. L. and Kovac, A. (2001b). Local extremes, runs, strings and multiresolution. Ann. Statist. 29 61–65.
  • DeVore, R. A. (1998). Nonlinear approximation. Acta Numerica 7 51–150.
  • Dobson, D. C. and Vogel, C. R. (1997). Convergence of an iterative method for total variation denoising. SIAM J. Numer. Anal. 34 1779–1791.
  • Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over p-balls for q-error. Probab. Theory Related Fields 99 277–303.
  • Friedman, J., Hastie, T., Höfling, H. and Tibshirani, R. (2007). Pathways coordinate optimization. Ann. Appl. Stat. 1 302–332.
  • Greenshtein, E. (2006). Best subset selection, persistence in high-dimensional statistical learning and optimization under l1 constraint. Ann. Statist. 34 2367–2386.
  • Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes 23. Springer, Berlin.
  • Lorentz, G. G., Golitschek, M. V. and Makovoz, Y. (1996). Constructive Approximation: Advanced Problems. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 304. Springer, Berlin.
  • Loubes, J. M. and van de Geer, S. (2002). Adaptive estimation in regression, using soft thresholding type penalties. Statistica Neerlandica 56 454–479.
  • Mammen, E. and van de Geer, S. (1997). Locally adaptive regression splines. Ann. Statist. 25 387–413.
  • Massart, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Mathematics 1896. Springer, Berlin.
  • Rudin, L. I., Osher, S. and Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D 60 259–268.
  • Stanley, R. P. (2000). Enumerative Combinatorics. Cambridge Univ. Press, Cambridge.
  • van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press, Cambridge.
  • van de Geer, S. (2007). Oracle Inequalities and Regularization 191–252. European Mathematical Society, Zürich.
  • van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence of Empirical Processes. Springer, New York.
  • Wainwright, M. J. (2006). Sharp thresholds for high-dimensional and noisy recovery of sparsity. IEEE Trans. Info. Theory. To appear.
  • Zhao, P. and Yu, B. (2006). On model selection consistency of lasso. J. Mach. Learn. Res. 7 2541–2563.