Institute of Mathematical Statistics Collections

Testing earthquake predictions

Brad Luen and Philip B. Stark

Full-text: Open access


Statistical tests of earthquake predictions require a null hypothesis to model occasional chance successes. To define and quantify ‘chance success’ is knotty. Some null hypotheses ascribe chance to the Earth: Seismicity is modeled as random. The null distribution of the number of successful predictions – or any other test statistic – is taken to be its distribution when the fixed set of predictions is applied to random seismicity. Such tests tacitly assume that the predictions do not depend on the observed seismicity. Conditioning on the predictions in this way sets a low hurdle for statistical significance. Consider this scheme: When an earthquake of magnitude 5.5 or greater occurs anywhere in the world, predict that an earthquake at least as large will occur within 21 days and within an epicentral distance of 50 km. We apply this rule to the Harvard centroid-moment-tensor (CMT) catalog for 2000–2004 to generate a set of predictions. The null hypothesis is that earthquake times are exchangeable conditional on their magnitudes and locations and on the predictions–a common “nonparametric” assumption in the literature. We generate random seismicity by permuting the times of events in the CMT catalog. We consider an event successfully predicted only if (i) it is predicted and (ii) there is no larger event within 50 km in the previous 21 days. The P-value for the observed success rate is <0.001: The method successfully predicts about 5% of earthquakes, far better than ‘chance,’ because the predictor exploits the clustering of earthquakes – occasional foreshocks – which the null hypothesis lacks. Rather than condition on the predictions and use a stochastic model for seismicity, it is preferable to treat the observed seismicity as fixed, and to compare the success rate of the predictions to the success rate of simple-minded predictions like those just described. If the proffered predictions do no better than a simple scheme, they have little value.

Chapter information

Deborah Nolan and Terry Speed, eds., Probability and Statistics: Essays in Honor of David A. Freedman (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2008), 302-315

First available in Project Euclid: 7 April 2008

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62-06: Proceedings, conferences, collections, etc. 62G10: Hypothesis testing 62P12: Applications to environmental and related topics 62P35: Applications to physics 62P99: None of the above, but in this section 86-06: Proceedings, conferences, collections, etc. 86A15: Seismology 86A17: Global dynamics, earthquake problems

aftershocks earthquake clustering stochastic processes

Copyright © 2008, Institute of Mathematical Statistics


Luen, Brad; Stark, Philip B. Testing earthquake predictions. Probability and Statistics: Essays in Honor of David A. Freedman, 302--315, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2008. doi:10.1214/193940307000000509.

Export citation


  • [1] Aceves, R., Park, S. and Strauss, D. (1996). Statistical evaluation of the VAN method using the historic earthquake catalog in Greece. Geophys. Res. Lett. 23 1425–1428.
  • [2] Arabelos, D., Asteriadis, G., Contadakis, M., Zioutas, G., Xu, D., Zhang, C. and Zheng, B. (2001). The use of an outlier detecting method in time series of continuous daily measurements of underground water level and temperature in earthquake prediction investigation. Tectonophysics 338 315–323.
  • [3] Bleier, T. and Freund, F. (2005). Earthquake alarm. IEEE Spectrum
  • [4] Bodri, B. (2001). A neural-network model for earthquake occurrence. J. Geodynamics 32 289–310.
  • [5] Bolt, B. (1993). Earthquakes. W. H. Freeman, New York.
  • [6] Bowman, D., Ouillon, G., Sammis, C., Sornette, A. and Sornette, D. (1998). An observational test of the critical earthquake concept. J. Geophys. Res. 103 24,359–24,372.
  • [7] Buskirk, R. E., Frohlich, C. and Latham, G. V. (1981). Unusual animal behavior before earthquakes: A review of possible sensory mechanisms. Rev. Geophys. Space Phys. 19 247–270.
  • [8] Console, R. (2001). Testing earthquake forecast hypotheses. Tectonophysics 338 261–268.
  • [9] Console, R., Pantosti, D. and D’Addezio, G. (2002). Probabilistic approach to earthquake prediction. Ann. Geophys. 45 723–731.
  • [10] Evison, F. F. and Rhoades, D. A. (1993). The precursory earthquake swarm in New Zealand: Hypothesis tests. New Zealand J. of Geol. and Geophys. 36 51–60.
  • [11] Evison, F. F. and Rhoades, D. A. (1997). The precursory earthquake swarm in New Zealand: Hypothesis tests. II. New Zealand J. of Geol. and Geophys. 40 537–547.
  • [12] Freedman, D. (1999). From association to causation: Some remarks on the history of statistics. Statist. Sci. 14 243–258.
  • [13] Freedman, D. and Stark, P. (2003). What is the Chance of an Earthquake? NATO Science Series IV: Earth and Environmental Sciences 32 201–213. Kluwer, Dordrecht, The Netherlands.
  • [14] Geller, R., Braginski, A. and Campbell, W. (2006). Earthquake precursors or background noise? Available at apr06/3275.
  • [15] Gerstenberger, M., Wiemer, S., Jones, L. and Reasenberg, P. (2005). Real-time forecasts of tomorrow’s earthquakes in California. Nature 435 328–331.
  • [16] Holliday, J., Rundle, J., Tiampo, K., Klein, W. and Donnellan, A. (2006). Systematic procedural and sensitivity analysis of the pattern informatics method for forecasting large (m>5) earthquake events in southern California. Pure Appl. Geoph. 163 2433–2454.
  • [17] Honkura, Y. and Tanaka, N. (1996). Probability of earthquake occurrence in Greece with special reference to the VAN predictions. Geophys. Res. Lett. 23 1417–1420.
  • [18] Ikeya, M. (2004). Earthquakes and Animals: From Folk Legends to Science. World Scientific Publishing, Hackensack, NJ.
  • [19] Jackson, D. (1996). Hypothesis testing and earthquake prediction. Proc. Natl. Acad. Sci. 93 3772–3775.
  • [20] Kagan, Y. (1996). VAN earthquake predictions – an attempt at statistical evaluation. Geophys. Res. Lett. 23 1315–1318.
  • [21] Kagan, Y. and Jackson, D. (1994). Long-term probabilistic forecasting of earthquakes. J. Geophys. Res. 99 13,685–13,700.
  • [22] Kagan, Y. and Jackson, D. (1995). New seismic gap hypothesis: Five years after. J. Geophys. Res. 100 3943–3959.
  • [23] Keilis-Borok, V., Knopoff, L. and Rotvain, I. (1980). Bursts of aftershocks, long-term precursors of strong earthquakes. Nature 283 259–263.
  • [24] Keilis-Borok, V. I. and Kossobokov, V. G. (1990). Premonitory activation of earthquake flow: Algorithm M8. Physics of the Earth and Planetary Interiors 61 73–83.
  • [25] Kerr, R. (1980). Quake prediction by animals gaining respect. Science 208 695–696.
  • [26] Kirschvink, J. (2000). Earthquake prediction by animals: evolution and sensory perception. Bull. Seis. Soc. Am. 90 312–323.
  • [27] Knopoff, L. (2000). The magnitude distribution of declustered earthquakes in southern California. Proc. Natl. Acad. Sci. 97 11,880–11,884.
  • [28] Kossobokov, V., Romashkova, L., Keilis-Borok, V. and Healy, J. (1999). Testing earthquake prediction algorithms: statistically significant advance prediction of the largest earthquakes in the Circum-Pacific, 1992–1997. Phys. Earth Planet. Inter. 11 187–196.
  • [29] Michael, A. (1996). The evaluation of VLF guided waves as possible earthquake precursors. Technical Report 96-67, USGS, Menlo Park, CA.
  • [30] Morgan, F., Williams, E. and Madden, T. (1989). Streaming potential properties of westerly granite with applications. J. Geophys. Res. 94 12,449–12,461.
  • [31] Mulargia, F. and Gasperini, P. (1992). Evaluating the statistical validity beyond chance of ‘VAN’ earthquake precursors. Geophys. J. Intl. 111 32–44.
  • [32] Nanjo, K., Holliday, J., c. Chen, C., Rundle, J. and Turcotte, D. (2006). Application of a modified pattern informatics method to forecasting the locations of future large earthquakes in the central Japan. Tectonophysics 424 351–366.
  • [33] Ogata, Y. (1988). Statistical models for earthquake occurrences and residual analysis for point processes. J. Amer. Statist. Assoc. 83 9–27.
  • [34] Ogata, Y. (1999). Seismicity analysis through point-process modeling: A review. Pure Appl. Geophys. 155 471–507.
  • [35] Ogata, Y. and Zhuang, J. (2006). Space-time ETAS models and an improved extension. Tectonophysics 413 13–23.
  • [36] Poitrasson, F., Dundas, S., Toutain, J., Munoz, M. and Rigo, A. (1999). Earthquake-related elemental and isotopic lead anomaly in a springwater. Earth. Planet. Sci. Lett. 169 269–276.
  • [37] Prozorov, A. (1988). Effect of an earthquake occurrence on the flow intensity of the subsequent events. Pure and Applied Geophysics 127 1–19.
  • [38] Reasenberg, P. (1985). Second-order moment of central California seismicity, 1969–1982. J. Geophys. Res. 90 5479–5496.
  • [39] Riedel, K. (1996). Statistical tests for evaluating earthquake prediction methods. Geophys. Res. Lett. 23 1407–1409.
  • [40] Shi, Y., Liu, J. and Zhang, G. (2001). An evaluation of Chinese annual earthquake predictions, 1990–1998. J. Appl. Probab. 38A 222–231.
  • [41] Shou, Z. (1999). Earthquake clouds: A reliable precursor. Science and Utopia 64 53–57.
  • [42] Stark, P. (1996). A few considerations for ascribing statistical significance to earthquake predictions. Geophys. Res. Lett. 23 1399–1402.
  • [43] Stark, P. (1997). Earthquake prediction: The null hypothesis. Geophys. J. Intl. 131 495–499.
  • [44] Stark, P. (1997-2007). SticiGui: Statistics tools for internet and classroom instruction with a graphical user interface. Available at http://statistics.berkeley. edu/~stark/SticiGui.
  • [45] Stark, P. and Frohlich, C. (1985). The depths of the deepest deep earthquakes. J. Geophys. Res. 90 1859–1869.
  • [46] Stothers, R. (2004). Earthquake prediction in antiquity. Acient Hist. Bull. 18 101–108.
  • [47] Tiampo, K., Rundle, J., McGinnis, S., Gross, S. and Klein, W. (2002). Mean-field threshold systems and phase dynamics: An application to earthquake fault systems. Europhys. Lett. 60 481–488.
  • [48] Udias, A. and Rice, J. (1975). Statistical analysis of microearthquake activity near San Andreas Geophysical Observatory. Bull. Seis. Soc. Am. 65 809–828.
  • [49] Utada, H. (1996). Difficulty of statistical evaluation of an earthquake prediction method. Geophys. Res. Lett. 23 1391–1394.
  • [50] Varotsos, P., Alexopoulos, K. and Nomicos, K. (1981). Seismic electric currents. Prakt. Akad. Athenon 56 277–286.
  • [51] Varotsos, P., Eftaxias, K., Vallianatos, F. and Lazaridou, M. (1996). Basic principles for evaluating an earthquake prediction method. Geophys. Res. Lett. 23 1295–1298.
  • [52] Vere-Jones, D. (1970). Stochastic models for earthquake occurrence. J. Roy. Statist. Soc. Ser. B 32 1–62.
  • [53] Wang, C. Y., Goodman, R. E., Sundaram, P. N. and Morrison, H. F. (1975). Electrical resistivity of granite in frictional sliding: Application to earthquake prediction. Geophys. Res. Lett. 2 525–528.
  • [54] Wyss, M. and Allmann, A. (1996). Probability of chance correlations of earthquakes with predictions in areas of heterogeneous seismicity rate: The VAN case. Geophys. Res. Lett. 23 1307–1310.
  • [55] Wyss, M. and Burford, R. (1987). Occurrence of a predicted earthquake on the San Andreas fault. Nature 329 323–325.
  • [56] Zhuang, J., Chang, C.-P., Ogata, Y. and Chen, Y.-I. (2005). A study on the background and clustering seismicity in the Taiwan region by using point process models. J. Geophys. Res. 110 B05S18.
  • [57] Zöller, G., Hainzl, S. and Kurths, J. (2001). Observation of growing correlation length as an indicator for critical point behavior prior to large earthquakes. J. Geophys. Res. 106 2167–2176.