The Annals of Statistics

Adaptive multiscale detection of filamentary structures in a background of uniform random points

Ery Arias-Castro, David L. Donoho, and Xiaoming Huo

Full-text: Open access

Abstract

We are given a set of n points that might be uniformly distributed in the unit square [0,1]2. We wish to test whether the set, although mostly consisting of uniformly scattered points, also contains a small fraction of points sampled from some (a priori unknown) curve with Cα-norm bounded by β. An asymptotic detection threshold exists in this problem; for a constant T(α,β)>0, if the number of points sampled from the curve is smaller than T(α,β)n1/(1+α), reliable detection is not possible for large n. We describe a multiscale significant-runs algorithm that can reliably detect concentration of data near a smooth curve, without knowing the smoothness information α or β in advance, provided that the number of points on the curve exceeds T*(α,β)n1/(1+α). This algorithm therefore has an optimal detection threshold, up to a factor T*/T.

At the heart of our approach is an analysis of the data by counting membership in multiscale multianisotropic strips. The strips will have area 2/n and exhibit a variety of lengths, orientations and anisotropies. The strips are partitioned into anisotropy classes; each class is organized as a directed graph whose vertices all are strips of the same anisotropy and whose edges link such strips to their “good continuations.” The point-cloud data are reduced to counts that measure membership in strips. Each anisotropy graph is reduced to a subgraph that consist of strips with significant counts. The algorithm rejects H0 whenever some such subgraph contains a path that connects many consecutive significant counts.

Article information

Source
Ann. Statist., Volume 34, Number 1 (2006), 326-349.

Dates
First available in Project Euclid: 2 May 2006

Permanent link to this document
https://projecteuclid.org/euclid.aos/1146576265

Digital Object Identifier
doi:10.1214/009053605000000787

Mathematical Reviews number (MathSciNet)
MR2275244

Zentralblatt MATH identifier
1091.62095

Subjects
Primary: 62M30: Spatial processes
Secondary: 62G10: Hypothesis testing 62G20: Asymptotic properties

Keywords
Multiscale geometric analysis pattern recognition good continuation Erdös–Rényi laws runs test beamlets

Citation

Arias-Castro, Ery; Donoho, David L.; Huo, Xiaoming. Adaptive multiscale detection of filamentary structures in a background of uniform random points. Ann. Statist. 34 (2006), no. 1, 326--349. doi:10.1214/009053605000000787. https://projecteuclid.org/euclid.aos/1146576265


Export citation

References

  • Abramowicz, H., Horn, D., Naftaly, U. and Sahar-Pikielny, C. (1997). An orientation-selective neural network for pattern identification in particle detectors. In Advances in Neural Information Processing Systems 9 (M. Mozer, M. I. Jordan and T. Petsche, eds.) 925–931. MIT Press, Cambridge, MA.
  • Aho, A. V., Hopcroft, J. E. and Ullman, J. D. (1983). Data Structures and Algorithms. Addison–Wesley, Reading, MA.
  • Arias-Castro, E. (2004). Graphical structures for geometric detection. Ph.D. dissertation, Stanford Univ.
  • Arias-Castro, E., Donoho, D. L. and Huo, X. (2005). Near-optimal detection of geometric objects by fast multiscale methods. IEEE Trans. Inform. Theory 51 2402–2425.
  • Arias-Castro, E., Donoho, D. L., Huo, X. and Tovey, C. (2005). Connect-the-dots: How many random points can a regular curve pass through? Adv. in Appl. Probab. 37 571–603.
  • Arratia, R. and Waterman, M. S. (1989). The Erdös–Rényi strong law for pattern matching with a given proportion of mismatches. Ann. Probab. 17 1152–1169.
  • Buhmann, J. M., Malik, J. and Perona, P. (1999). Image recognition: Visual grouping, recognition, and learning. Proc. Natl. Acad. Sci. USA 96 14,203–14,204.
  • Copeland, A. C., Ravichandran, G. and Trivedi, M. M. (1995). Localized Radon transform-based detection of ship wakes in SAR images. IEEE Trans. Geoscience and Remote Sensing 33 35–45.
  • Courtney, S. M. and Ungerleider, L. G. (1997). What fMRI has taught us about human vision. Current Opinion in Neurobiology 7 554–561.
  • David, G. and Semmes, S. (1993). Analysis of and on Uniformly Rectifiable Sets. Amer. Math. Soc., Providence, RI.
  • Desolneux, A., Moisan, L. and Morel, J.-M. (2000). Meaningful alignments. Internat. J. Computer Vision 40 7–23.
  • Desolneux, A., Moisan, L. and Morel, J.-M. (2003). A grouping principle and four applications. IEEE Trans. Pattern Analysis and Machine Intelligence 25 508–513.
  • Desolneux, A., Moisan, L. and Morel, J.-M. (2003). Maximal meaningful events and applications to image analysis. Ann. Statist. 31 1822–1851.
  • Donoho, D. L. (1997). CART and best-ortho-basis: A connection. Ann. Statist. 25 1870–1911.
  • Donoho, D. L. (1999). Wedgelets: Nearly minimax estimation of edges. Ann. Statist. 27 859–897.
  • Donoho, D. L. and Huo, X. (2002). Beamlets and multiscale image analysis. In Multiscale and Multiresolution Methods. Lecture Notes Comput. Sci. Eng. 20 149–196. Springer, Berlin.
  • Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200–1224.
  • Donoho, D. L. and Levi, O. (2004). Fast X-ray and beamlet transforms for three-dimensional data. In Modern Signal Processing (D. N. Rockmore and D. M. Healy, Jr., eds.) 79–116. Cambridge Univ. Press.
  • Field, D., Hayes, A. and Hess, R. (1993). Contour integration by the human visual system: Evidence for a local “association field.” Vision Research 33 173–193.
  • Ho, M.-W. (2004). In search of the sublime. Institute of Science in Society. Available at www.i-sis.org.uk/sublime.php.
  • Huo, X., Chen, J. and Donoho, D. L. (2003). Multiscale detection of filamentary features in image data. In Wavelets: Applications in Signal and Image Processing X (M. A. Unser, A. Aldroubi and A. F. Laine, eds.) 592–606. SPIE, Bellingham, WA.
  • Huo, X., Chen, J. and Donoho, D. L. (2003). Multiscale significance run: Realizing the “most powerful” detection in noisy images. In Proc. Thirty Seventh Asilomar Conference on Signals, Systems, and Computers 1 321–326. IEEE, Piscataway, NJ.
  • Huo, X., Donoho, D. L., Tovey, C. and Arias-Castro, E. (2004). Dynamic programming methods for “connecting the dots” in scattered point sets. Technical report, Dept. Statistics, Stanford Univ.
  • Jones, P. W. (1990). Rectifiable sets and the traveling salesman problem. Invent. Math. 102 1–15.
  • Kovacs, I. and Julesz, B. (1993). A closed curve is much more than an incomplete one: Effect of closure in figure-ground segementation. Proc. Natl. Acad. Sci. USA 90 7495–7497.
  • Legge, G. E., Kersten, D. and Burgess, A. E. (1987). Contrast discrimination in noise. J. Opt. Soc. Amer. A 4 391–404.
  • Lerman, G. (2003). Quantifying curvelike structures of measures by using $L_2$ Jones quantities. Comm. Pure Appl. Math. 56 1294–1365.
  • Levi, D. M. and Klein, S. A. (2000). Seeing circles: What limits shape perception? Vision Research 40 2329–2339.
  • Mendola, J. D., Dale, A. M., Fischl, B., Liu, A. K. and Tootell, R. B. H. (1999). The representation of illusory and real contours in human cortical visual areas revealed by functional magnetic resonance imaging. J. Neuroscience 19 8560–8572.
  • Pizlo, Z., Salach-Golyska, M. and Rosenfeld, A. (1997). Curve detection in a noisy image. Vision Research 37 1217–1241.
  • Qaddoumi, N., Ranu, E., McColskey, J. D., Mirshahi, R. and Zoughi, R. (2000). Microwave detection of stress-induced fatigue cracks in steel and potential for crack opening determination. Research in Nondestructive Evaluation 12 87–103.
  • Sharon, E., Brandt, A. and Basri, R. (2000). Fast multiscale image segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 1 70–77.
  • Small, C. G. (1996). The Statistical Theory of Shape. Springer, Berlin.
  • Tupin, F., Maitre, H., Mangin, J.-F., Nicolas, J.-M. and Pechersky, E. (1998). Detection of linear features in SAR images: Application to road network extraction. IEEE Trans. Geoscience and Remote Sensing 36 434–453.
  • Wertheimer, M. (1938). Laws of Organization in Perceptual Forms. Harcourt Brace, London.