Statistical multiresolution Dantzig estimation in imaging: Fundamental concepts and algorithmic framework
Klaus Frick, Philipp Marnitz, and Axel Munk
Source: Electron. J. Statist. Volume 6
(2012), 231-268.
Abstract
In this paper we are concerned with fully automatic and locally adaptive estimation of functions in a “signal + noise”-model where the regression function may additionally be blurred by a linear operator, e.g. by a convolution. To this end, we introduce a general class of statistical multiresolution estimators and develop an algorithmic framework for computing those. By this we mean estimators that are defined as solutions of convex optimization problems with ℓ∞-type constraints. We employ a combination of the alternating direction method of multipliers with Dykstra’s algorithm for computing orthogonal projections onto intersections of convex sets and prove numerical convergence. The capability of the proposed method is illustrated by various examples from imaging and signal detection.
First Page:
Show
Hide
Keywords: alternating direction method of multipliers (ADMM); biophotonics; Dantzig selector; Dykstra’s projection algorithm; local adaption; signal detection; statistical imaging; statistical multiscale analysis; statistical regularization
Full-text: Open access
Links and Identifiers
Permanent link to this document: http://projecteuclid.org/euclid.ejs/1330524559
Digital Object Identifier: doi:10.1214/12-EJS671
Mathematical Reviews number (MathSciNet): MR2988407
References
[1] Aujol, J.-F., Aubert, G., Blanc-Féraud, L. and Chambolle, A. (2005). Image decomposition into a bounded variation component and an oscillating component., J. Math. Imaging Vision 22 71–88.
Mathematical Reviews (MathSciNet): MR2138585
Digital Object Identifier: doi:10.1007/s10851-005-4783-8
[2] Becker, S., Candès, E. and Grant, M. (2011). Templates for convex cone problems with applications to sparse signal recovery., Math. Program. Comput. 3 165-218.
Mathematical Reviews (MathSciNet): MR2833262
Zentralblatt MATH: 06110466
Digital Object Identifier: doi:10.1007/s12532-011-0029-5
[3] Bertalmio, M., Caselles, V., Rougé, B. and Solé, A. (2003). TV based image restoration with local constraints., J. Sci. Comput. 19 95–122. Special issue in honor of the sixtieth birthday of Stanley Osher.
Mathematical Reviews (MathSciNet): MR2028838
Zentralblatt MATH: 1034.49036
Digital Object Identifier: doi:10.1023/A:1025391506181
[4] Bickel, P. J., Ritov, Y. and Tsybakov, A. B. (2009). Simultaneous analysis of lasso and Dantzig selector., Ann. Statist. 37 1705–1732.
Mathematical Reviews (MathSciNet): MR2533469
Zentralblatt MATH: 1173.62022
Digital Object Identifier: doi:10.1214/08-AOS620
Project Euclid: euclid.aos/1245332830
[5] Boyle, J. P. and Dykstra, R. L. (1986). A method for finding projections onto the intersection of convex sets in Hilbert spaces. In, Advances in order restricted statistical inference (Iowa City, Iowa, 1985). Lecture Notes in Statist. 37 28–47. Springer, Berlin.
Mathematical Reviews (MathSciNet): MR875647
Zentralblatt MATH: 0603.49024
Digital Object Identifier: doi:10.1007/978-1-4613-9940-7_3
[6] Boysen, L., Kempe, A., Liebscher, V., Munk, A. and Wittich, O. (2009). Consistencies and rates of convergence of jump-penalized least squares estimators., Ann. Statist. 37 157–183.
Mathematical Reviews (MathSciNet): MR2488348
Zentralblatt MATH: 1155.62034
Digital Object Identifier: doi:10.1214/07-AOS558
Project Euclid: euclid.aos/1232115931
[7] Candès, E. and Tao, T. (2007). The Dantzig selector: statistical estimation when, p is much larger than n. Ann. Statist. 35 2313–2351.
Mathematical Reviews (MathSciNet): MR2382644
Zentralblatt MATH: 1139.62019
Digital Object Identifier: doi:10.1214/009053606000001523
Project Euclid: euclid.aos/1201012958
[8] Chen, S. S., Donoho, D. L. and Saunders, M. A. (2001). Atomic Decomposition by Basis Pursuit., SIAM Review 43 129-159.
Mathematical Reviews (MathSciNet): MR1854649
Zentralblatt MATH: 0979.94010
Digital Object Identifier: doi:10.1137/S003614450037906X
[9] Csiszár, I. (1991). Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems., Ann. Statist. 19 2032–2066.
Mathematical Reviews (MathSciNet): MR1135163
Zentralblatt MATH: 0753.62003
Digital Object Identifier: doi:10.1214/aos/1176348385
Project Euclid: euclid.aos/1176348385
[10] Davies, P. L. and Kovac, A. (2001). Local extremes, runs, strings and multiresolution., Ann. Statist. 29 1–65. With discussion and rejoinder by the authors.
Mathematical Reviews (MathSciNet): MR1833958
Zentralblatt MATH: 1029.62038
Digital Object Identifier: doi:10.1214/aos/996986501
Project Euclid: euclid.aos/996986501
[11] Davies, P. L. and Kovac, A. (2004). Densities, spectral densities and modality., Ann. Statist. 32 1093–1136.
Mathematical Reviews (MathSciNet): MR2065199
Zentralblatt MATH: 1093.62042
Digital Object Identifier: doi:10.1214/009053604000000364
Project Euclid: euclid.aos/1085408496
[12] Davies, P. L., Kovac, A. and Meise, M. (2009). Nonparametric regression, confidence regions and regularization., Ann. Statist. 37 2597–2625.
Mathematical Reviews (MathSciNet): MR2541440
Zentralblatt MATH: 1173.62023
Digital Object Identifier: doi:10.1214/07-AOS575
Project Euclid: euclid.aos/1247836662
[13] Deutsch, F. and Hundal, H. (1994). The rate of convergence of Dykstra’s cyclic projections algorithm: the polyhedral case., Numer. Funct. Anal. Optim. 15 537–565.
Mathematical Reviews (MathSciNet): MR1281561
Digital Object Identifier: doi:10.1080/01630569408816580
[14] Dobson, D. C. and Vogel, C. R. (1997). Convergence of an iterative method for total variation denoising., SIAM J. Numer. Anal. 34 1779–1791.
Mathematical Reviews (MathSciNet): MR1472196
Zentralblatt MATH: 0898.65034
Digital Object Identifier: doi:10.1137/S003614299528701X
[15] Dong, Y., Hintermüller, M. and Rincon-Camacho, M. (May 2011). Automated Regularization Parameter Selection in Multi-Scale Total Variation Models for Image Restoration., J. Math. Imaging Vision 40 82-104(23).
Mathematical Reviews (MathSciNet): MR2782120
Zentralblatt MATH: 06127965
Digital Object Identifier: doi:10.1007/s10851-010-0248-9
[16] Dümbgen, L. and Johns, R. B. (2004). Confidence Bands for Isotonic Median Curves Using Sign Tests., J. Comput. Graph. Statist 13 519-533.
Mathematical Reviews (MathSciNet): MR2063998
Digital Object Identifier: doi:10.1198/1061860043506
[17] Dümbgen, L. and Spokoiny, V. G. (2001). Multiscale testing of qualitative hypotheses., Ann. Statist. 29 124–152.
Mathematical Reviews (MathSciNet): MR1833961
Zentralblatt MATH: 1029.62070
Digital Object Identifier: doi:10.1214/aos/996986504
Project Euclid: euclid.aos/996986504
[18] Dümbgen, L. and Walther, G. (2008). Multiscale inference about a density., Ann. Statist. 36 1758–1785.
Mathematical Reviews (MathSciNet): MR2435455
Zentralblatt MATH: 1142.62336
Digital Object Identifier: doi:10.1214/07-AOS521
Project Euclid: euclid.aos/1216237299
[19] Ekeland, I. and Temam, R. (1976)., Convex analysis and variational problems. Studies in Mathematics and its Applications 1. North-Holland Publishing Co., Amsterdam-Oxford.
Mathematical Reviews (MathSciNet): MR463994
[20] Fortin, M. and Glowinski, R. (1983)., Augmented Lagrangian methods. Studies in Mathematics and its Applications 15. North-Holland Publishing Co., Amsterdam. Applications to the numerical solution of boundary value problems, Translated from the French by B. Hunt and D. C. Spicer.
Mathematical Reviews (MathSciNet): MR724072
Zentralblatt MATH: 0525.65045
[21] Frick, K., Marnitz, P. and Munk, A. (2010). Shape Constrained Regularisation by Statistical Multiresolution for Inverse Problems: Asymptotic Analysis Technical Report. Available at, http://arxiv.org/abs/1003.3323.
Mathematical Reviews (MathSciNet): MR2924303
[22] Frick, K. and Scherzer, O. (2010). Regularization of ill-posed linear equations by the non-stationary Augmented Lagrangian Method., J. Integral Equations Appl. 22 217-257.
Mathematical Reviews (MathSciNet): MR2661720
Zentralblatt MATH: 1203.90156
Digital Object Identifier: doi:10.1216/JIE-2010-22-2-217
Project Euclid: euclid.jiea/1277125622
[23] Gaffke, N. and Mathar, R. (1989). A Cyclic Projection Algorithm Via Duality., Metrika 36 29–54.
Mathematical Reviews (MathSciNet): MR985010
Digital Object Identifier: doi:10.1007/BF02614077
[24] Grasmair, M. (2007). The equivalence of the taut string algorithm and BV-regularization., J. Math. Imaging Vision 27 59–66.
Mathematical Reviews (MathSciNet): MR2297998
Digital Object Identifier: doi:10.1007/s10851-006-9796-4
[25] Hawkins, D. M. and Wixley, R. A. J. (1986). A Note on the Transformation of Chi-Squared Variables to Normality., Amer.Statist. 40 296–298.
[26] Hell, S. W. (2007). Far-Field Optical Nanoscopy., Science 316 1153-1158.
[27] Hell, S. W. and Wichmann, J. (1994). Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy., Opt. Lett. 19 780–782.
[28] Hotz, T., Marnitz, P., Stichtenoth, R., Davies, L., Kabluchko, Z. and Munk, A. (2012). Locally adaptive image denoising by a statistical multiresolution criterion., Comput. Stat. Data An. 56 543 - 558.
Mathematical Reviews (MathSciNet): MR2853754
[29] James, G. M., Radchenko, P. and Lv, J. (2009). DASSO: connections between the Dantzig selector and lasso., J. R. Stat. Soc. Ser. B Stat. Methodol. 71 127–142.
Mathematical Reviews (MathSciNet): MR2655526
Zentralblatt MATH: 1231.62129
Digital Object Identifier: doi:10.1111/j.1467-9868.2008.00668.x
[30] Kabluchko, Z. (2011). Extremes of the standardized Gaussian noise., Stochastic Process. Appl. 121 515–533.
Mathematical Reviews (MathSciNet): MR2763094
Zentralblatt MATH: 1225.60084
Digital Object Identifier: doi:10.1016/j.spa.2010.11.007
[31] Kabluchko, Z. and Munk, A. (2009). Shao’s theorem on the maximum of standardized random walk increments for multidimensional arrays., ESAIM Probab. Stat. 13 409–416.
Mathematical Reviews (MathSciNet): MR2554963
Zentralblatt MATH: 1188.60014
Digital Object Identifier: doi:10.1051/ps:2008020
[32] Kaipio, J. and Somersalo, E. (2005)., Statistical and computational inverse problems. Applied Mathematical Sciences 160. Springer-Verlag, New York.
Mathematical Reviews (MathSciNet): MR2102218
Zentralblatt MATH: 1068.65022
[33] Lu, Z., Pong, T. K. and Zhang, Y. (2010). An Alternating Direction Method for Finding Dantzig Selectors Technical Report. Available at, http://arxiv.org/abs/1011.4604v1.
Mathematical Reviews (MathSciNet): MR2957852
[34] Mammen, E. and van de Geer, S. (1997). Locally adaptive regression splines., Ann. Statist. 25 387–413.
Mathematical Reviews (MathSciNet): MR1429931
Zentralblatt MATH: 0871.62040
Digital Object Identifier: doi:10.1214/aos/1034276635
Project Euclid: euclid.aos/1034276635
[35] Meyer, Y. (2001)., Oscillating patterns in image processing and nonlinear evolution equations. University Lecture Series 22. American Mathematical Society, Providence, RI. The fifteenth Dean Jacqueline B. Lewis memorial lectures.
Mathematical Reviews (MathSciNet): MR1852741
Zentralblatt MATH: 0987.35003
[36] Mildenberger, T. (2008). A geometric interpretation of the multiresolution criterion in nonparametric regression., J. Nonparametr. Stat. 20 1048-5252.
Mathematical Reviews (MathSciNet): MR2454614
Zentralblatt MATH: 1147.62037
Digital Object Identifier: doi:10.1080/10485250802360994
[37] Mohler, G. O., Bertozzi, A. L., Goldstein, T. A. and Osher, S. J. (2011). Fast TV Regularization for 2D Maximum Penalized Likelihood Estimation., J. Comput. Graph. Statist 20 479-491.
Mathematical Reviews (MathSciNet): MR2847805
Digital Object Identifier: doi:10.1198/jcgs.2010.09048
[38] Munk, A., Bissantz, N., Wagner, T. and Freitag, G. (2005). On difference-based variance estimation in nonparametric regression when the covariate is high dimensional., J. R. Stat. Soc. Ser. B Stat. Methodol. 67 19-41.
Mathematical Reviews (MathSciNet): MR2136637
Zentralblatt MATH: 1060.62047
Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00486.x
[39] Pawley, J. B. (2006)., Handbook of Biological Confocal Microscopy. Springer.
[40] Polzehl, J. and Spokoiny, V. G. (2000). Adaptive weights smoothing with applications to image restoration., J. R. Stat. Soc. Ser. B Stat. Methodol. 62 335–354.
Mathematical Reviews (MathSciNet): MR1749543
Digital Object Identifier: doi:10.1111/1467-9868.00235
[41] Romberg, J. K. (2008). The Dantzig selector and generalized thresholding. In, CISS 22-25. IEEE.
[42] Siegmund, D. and Yakir, B. (2000). Tail probabilities for the null distribution of scanning statistics., Bernoulli 6 191–213.
Mathematical Reviews (MathSciNet): MR1748719
Digital Object Identifier: doi:10.2307/3318574
Project Euclid: euclid.bj/1081788026
[43] Tibshirani, R. (1994). Regression Shrinkage and Selection Via the Lasso., J. R. Stat. Soc. Ser. B Stat. Methodol. 58 267–288.
Mathematical Reviews (MathSciNet): MR1379242
[44] Vogel, C. R. (2002)., Computational methods for inverse problems. Frontiers in Applied Mathematics 23. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA. With a foreword by H. T. Banks.
Mathematical Reviews (MathSciNet): MR1928831
Zentralblatt MATH: 1008.65103
[45] Wang, Z., Bovik, A. C., Sheikh, H. R. and Simoncelli, E. P. (2004). Image Quality Assessment: From Error Visibility to Structural Similarity., IEEE Trans. Image Process. 13 600–612.
[46] Xu, S. (2000). Estimation of the convergence rate of Dykstra’s cyclic projections algorithm in polyhedral case., Acta Math. Appl. Sinica (English Ser.) 16 217–220.
Mathematical Reviews (MathSciNet): MR1774004
Digital Object Identifier: doi:10.1007/BF02677683
Electronic Journal of Statistics
