## Institute of Mathematical Statistics Collections

- Collections
- Volume 2, 2008, 302-315

### Testing earthquake predictions

#### Abstract

Statistical tests of earthquake predictions require a null hypothesis to model occasional chance successes. To define and quantify ‘chance success’ is knotty. Some null hypotheses ascribe chance to the Earth: Seismicity is modeled as random. The null distribution of the number of successful predictions – or any other test statistic – is taken to be its distribution when the fixed set of predictions is applied to random seismicity. Such tests tacitly assume that the predictions do not depend on the observed seismicity. Conditioning on the predictions in this way sets a low hurdle for statistical significance. Consider this scheme: When an earthquake of magnitude 5.5 or greater occurs anywhere in the world, predict that an earthquake at least as large will occur within 21 days and within an epicentral distance of 50 km. We apply this rule to the Harvard centroid-moment-tensor (CMT) catalog for 2000–2004 to generate a set of predictions. The null hypothesis is that earthquake times are exchangeable conditional on their magnitudes and locations and on the predictions–a common “nonparametric” assumption in the literature. We generate random seismicity by permuting the times of events in the CMT catalog. We consider an event successfully predicted only if (i) it is predicted and (ii) there is no larger event within 50 km in the previous 21 days. The *P*-value for the observed success rate is <0.001: The method successfully predicts about 5% of earthquakes, far better than ‘chance,’ because the predictor exploits the clustering of earthquakes – occasional foreshocks – which the null hypothesis lacks. Rather than condition on the predictions and use a stochastic model for seismicity, it is preferable to treat the observed seismicity as fixed, and to compare the success rate of the predictions to the success rate of simple-minded predictions like those just described. If the proffered predictions do no better than a simple scheme, they have little value.

#### Chapter information

**Source***Probability and Statistics: Essays in Honor of David A. Freedman* (Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2008)

**Dates**

First available in Project Euclid: 7 April 2008

**Permanent link to this document**

https://projecteuclid.org/euclid.imsc/1207580090

**Digital Object Identifier**

doi:10.1214/193940307000000509

**Mathematical Reviews number (MathSciNet)**

MR2459956

**Zentralblatt MATH identifier**

1166.62095

**Subjects**

Primary: 62-06: Proceedings, conferences, collections, etc. 62G10: Hypothesis testing 62P12: Applications to environmental and related topics 62P35: Applications to physics 62P99: None of the above, but in this section 86-06: Proceedings, conferences, collections, etc. 86A15: Seismology 86A17: Global dynamics, earthquake problems

**Keywords**

aftershocks earthquake clustering stochastic processes

**Rights**

Copyright © 2008, Institute of Mathematical Statistics

#### Citation

Luen, Brad; Stark, Philip B. Testing earthquake predictions. Probability and Statistics: Essays in Honor of David A. Freedman, 302--315, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2008. doi:10.1214/193940307000000509. https://projecteuclid.org/euclid.imsc/1207580090

#### References

- [1] Aceves, R., Park, S. and Strauss, D. (1996). Statistical evaluation of the VAN method using the historic earthquake catalog in Greece.
*Geophys. Res. Lett.***23**1425–1428. - [2] Arabelos, D., Asteriadis, G., Contadakis, M., Zioutas, G., Xu, D., Zhang, C. and Zheng, B. (2001). The use of an outlier detecting method in time series of continuous daily measurements of underground water level and temperature in earthquake prediction investigation.
*Tectonophysics***338**315–323. - [3] Bleier, T. and Freund, F. (2005). Earthquake alarm.
*IEEE Spectrum*http://www.spectrum.ieee.org/dec05/2367. - [4] Bodri, B. (2001). A neural-network model for earthquake occurrence.
*J. Geodynamics***32**289–310. - [5] Bolt, B. (1993).
*Earthquakes*. W. H. Freeman, New York. - [6] Bowman, D., Ouillon, G., Sammis, C., Sornette, A. and Sornette, D. (1998). An observational test of the critical earthquake concept.
*J. Geophys. Res.***103**24,359–24,372. - [7] Buskirk, R. E., Frohlich, C. and Latham, G. V. (1981). Unusual animal behavior before earthquakes: A review of possible sensory mechanisms.
*Rev. Geophys. Space Phys.***19**247–270. - [8] Console, R. (2001). Testing earthquake forecast hypotheses.
*Tectonophysics***338**261–268. - [9] Console, R., Pantosti, D. and D’Addezio, G. (2002). Probabilistic approach to earthquake prediction.
*Ann. Geophys.***45**723–731. - [10] Evison, F. F. and Rhoades, D. A. (1993). The precursory earthquake swarm in New Zealand: Hypothesis tests.
*New Zealand J. of Geol. and Geophys.***36**51–60. - [11] Evison, F. F. and Rhoades, D. A. (1997). The precursory earthquake swarm in New Zealand: Hypothesis tests. II.
*New Zealand J. of Geol. and Geophys.***40**537–547. - [12] Freedman, D. (1999). From association to causation: Some remarks on the history of statistics.
*Statist. Sci.***14**243–258.Mathematical Reviews (MathSciNet): MR1729836Digital Object Identifier: doi:10.1214/ss/1009212409Project Euclid: euclid.ss/1009212409Zentralblatt MATH: 1059.62501 - [13] Freedman, D. and Stark, P. (2003).
*What is the Chance of an Earthquake? NATO Science Series IV*:*Earth and Environmental Sciences***32**201–213. Kluwer, Dordrecht, The Netherlands. - [14] Geller, R., Braginski, A. and Campbell, W. (2006). Earthquake precursors or background noise? Available at http://spectrum.ieee.org/ apr06/3275.
- [15] Gerstenberger, M., Wiemer, S., Jones, L. and Reasenberg, P. (2005). Real-time forecasts of tomorrow’s earthquakes in California.
*Nature***435**328–331. - [16] Holliday, J., Rundle, J., Tiampo, K., Klein, W. and Donnellan, A. (2006). Systematic procedural and sensitivity analysis of the pattern informatics method for forecasting large (
*m*>5) earthquake events in southern California.*Pure Appl. Geoph.***163**2433–2454. - [17] Honkura, Y. and Tanaka, N. (1996). Probability of earthquake occurrence in Greece with special reference to the VAN predictions.
*Geophys. Res. Lett.***23**1417–1420. - [18] Ikeya, M. (2004).
*Earthquakes and Animals*:*From Folk Legends to Science*. World Scientific Publishing, Hackensack, NJ. - [19] Jackson, D. (1996). Hypothesis testing and earthquake prediction.
*Proc. Natl. Acad. Sci.***93**3772–3775. - [20] Kagan, Y. (1996). VAN earthquake predictions – an attempt at statistical evaluation.
*Geophys. Res. Lett.***23**1315–1318. - [21] Kagan, Y. and Jackson, D. (1994). Long-term probabilistic forecasting of earthquakes.
*J. Geophys. Res.***99**13,685–13,700. - [22] Kagan, Y. and Jackson, D. (1995). New seismic gap hypothesis: Five years after.
*J. Geophys. Res.***100**3943–3959. - [23] Keilis-Borok, V., Knopoff, L. and Rotvain, I. (1980). Bursts of aftershocks, long-term precursors of strong earthquakes.
*Nature***283**259–263. - [24] Keilis-Borok, V. I. and Kossobokov, V. G. (1990). Premonitory activation of earthquake flow: Algorithm M8.
*Physics of the Earth and Planetary Interiors***61**73–83. - [25] Kerr, R. (1980). Quake prediction by animals gaining respect.
*Science***208**695–696. - [26] Kirschvink, J. (2000). Earthquake prediction by animals: evolution and sensory perception.
*Bull. Seis. Soc. Am.***90**312–323. - [27] Knopoff, L. (2000). The magnitude distribution of declustered earthquakes in southern California.
*Proc. Natl. Acad. Sci.***97**11,880–11,884. - [28] Kossobokov, V., Romashkova, L., Keilis-Borok, V. and Healy, J. (1999). Testing earthquake prediction algorithms: statistically significant advance prediction of the largest earthquakes in the Circum-Pacific, 1992–1997.
*Phys. Earth Planet. Inter.***11**187–196. - [29] Michael, A. (1996). The evaluation of VLF guided waves as possible earthquake precursors. Technical Report 96-67, USGS, Menlo Park, CA.
- [30] Morgan, F., Williams, E. and Madden, T. (1989). Streaming potential properties of westerly granite with applications.
*J. Geophys. Res.***94**12,449–12,461. - [31] Mulargia, F. and Gasperini, P. (1992). Evaluating the statistical validity beyond chance of ‘VAN’ earthquake precursors.
*Geophys. J. Intl.***111**32–44. - [32] Nanjo, K., Holliday, J., c. Chen, C., Rundle, J. and Turcotte, D. (2006). Application of a modified pattern informatics method to forecasting the locations of future large earthquakes in the central Japan.
*Tectonophysics***424**351–366. - [33] Ogata, Y. (1988). Statistical models for earthquake occurrences and residual analysis for point processes.
*J. Amer. Statist. Assoc.***83**9–27. - [34] Ogata, Y. (1999). Seismicity analysis through point-process modeling: A review.
*Pure Appl. Geophys.***155**471–507. - [35] Ogata, Y. and Zhuang, J. (2006). Space-time ETAS models and an improved extension.
*Tectonophysics***413**13–23. - [36] Poitrasson, F., Dundas, S., Toutain, J., Munoz, M. and Rigo, A. (1999). Earthquake-related elemental and isotopic lead anomaly in a springwater.
*Earth. Planet. Sci. Lett.***169**269–276. - [37] Prozorov, A. (1988). Effect of an earthquake occurrence on the flow intensity of the subsequent events.
*Pure and Applied Geophysics***127**1–19. - [38] Reasenberg, P. (1985). Second-order moment of central California seismicity, 1969–1982.
*J. Geophys. Res.***90**5479–5496. - [39] Riedel, K. (1996). Statistical tests for evaluating earthquake prediction methods.
*Geophys. Res. Lett.***23**1407–1409. - [40] Shi, Y., Liu, J. and Zhang, G. (2001). An evaluation of Chinese annual earthquake predictions, 1990–1998.
*J. Appl. Probab.***38A**222–231.Mathematical Reviews (MathSciNet): MR1915547Zentralblatt MATH: 1014.86500Digital Object Identifier: doi:10.1239/jap/1085496604Project Euclid: euclid.jap/1085496604 - [41] Shou, Z. (1999). Earthquake clouds: A reliable precursor.
*Science and Utopia***64**53–57. - [42] Stark, P. (1996). A few considerations for ascribing statistical significance to earthquake predictions.
*Geophys. Res. Lett.***23**1399–1402. - [43] Stark, P. (1997). Earthquake prediction: The null hypothesis.
*Geophys. J. Intl.***131**495–499. - [44] Stark, P. (1997-2007). SticiGui: Statistics tools for internet and classroom instruction with a graphical user interface. Available at http://statistics.berkeley. edu/~stark/SticiGui.
- [45] Stark, P. and Frohlich, C. (1985). The depths of the deepest deep earthquakes.
*J. Geophys. Res.***90**1859–1869. - [46] Stothers, R. (2004). Earthquake prediction in antiquity.
*Acient Hist. Bull.***18**101–108. - [47] Tiampo, K., Rundle, J., McGinnis, S., Gross, S. and Klein, W. (2002). Mean-field threshold systems and phase dynamics: An application to earthquake fault systems.
*Europhys. Lett.***60**481–488. - [48] Udias, A. and Rice, J. (1975). Statistical analysis of microearthquake activity near San Andreas Geophysical Observatory.
*Bull. Seis. Soc. Am.***65**809–828. - [49] Utada, H. (1996). Difficulty of statistical evaluation of an earthquake prediction method.
*Geophys. Res. Lett.***23**1391–1394. - [50] Varotsos, P., Alexopoulos, K. and Nomicos, K. (1981). Seismic electric currents.
*Prakt. Akad. Athenon***56**277–286. - [51] Varotsos, P., Eftaxias, K., Vallianatos, F. and Lazaridou, M. (1996). Basic principles for evaluating an earthquake prediction method.
*Geophys. Res. Lett.***23**1295–1298. - [52] Vere-Jones, D. (1970). Stochastic models for earthquake occurrence.
*J. Roy. Statist. Soc. Ser. B***32**1–62.Mathematical Reviews (MathSciNet): MR272087JSTOR: links.jstor.org - [53] Wang, C. Y., Goodman, R. E., Sundaram, P. N. and Morrison, H. F. (1975). Electrical resistivity of granite in frictional sliding: Application to earthquake prediction.
*Geophys. Res. Lett.***2**525–528. - [54] Wyss, M. and Allmann, A. (1996). Probability of chance correlations of earthquakes with predictions in areas of heterogeneous seismicity rate: The VAN case.
*Geophys. Res. Lett.***23**1307–1310. - [55] Wyss, M. and Burford, R. (1987). Occurrence of a predicted earthquake on the San Andreas fault.
*Nature***329**323–325. - [56] Zhuang, J., Chang, C.-P., Ogata, Y. and Chen, Y.-I. (2005). A study on the background and clustering seismicity in the Taiwan region by using point process models.
*J. Geophys. Res.***110**B05S18. - [57] Zöller, G., Hainzl, S. and Kurths, J. (2001). Observation of growing correlation length as an indicator for critical point behavior prior to large earthquakes.
*J. Geophys. Res.***106**2167–2176.