Modern, powerful techniques for the residual analysis of spatial-temporal point process models are reviewed and compared. These methods are applied to California earthquake forecast models used in the Collaboratory for the Study of Earthquake Predictability (CSEP). Assessments of these earthquake forecasting models have previously been performed using simple, low-power means such as the L-test and N-test. We instead propose residual methods based on rescaling, thinning, superposition, weighted K-functions and deviance residuals. Rescaled residuals can be useful for assessing the overall fit of a model, but as with thinning and superposition, rescaling is generally impractical when the conditional intensity λ is volatile. While residual thinning and superposition may be useful for identifying spatial locations where a model fits poorly, these methods have limited power when the modeled conditional intensity assumes extremely low or high values somewhere in the observation region, and this is commonly the case for earthquake forecasting models. A recently proposed hybrid method of thinning and superposition, called super-thinning, is a more powerful alternative. The weighted K-function is powerful for evaluating the degree of clustering or inhibition in a model. Competing models are also compared using pixel-based approaches, such as Pearson residuals and deviance residuals. The different residual analysis techniques are demonstrated using the CSEP models and are used to highlight certain deficiencies in the models, such as the overprediction of seismicity in inter-fault zones for the model proposed by Helmstetter, Kagan and Jackson [Seismological Research Letters 78 (2007) 78–86], the underprediction of the model proposed by Kagan, Jackson and Rong [Seismological Research Letters 78 (2007) 94–98] in forecasting seismicity around the Imperial, Laguna Salada, and Panamint clusters, and the underprediction of the model proposed by Shen, Jackson and Kagan [Seismological Research Letters 78 (2007) 116–120] in forecasting seismicity around the Laguna Salada, Baja, and Panamint clusters.
Ann. Appl. Stat.
5(4):
2549-2571
(December 2011).
DOI: 10.1214/11-AOAS487
Adelfio, G. and Schoenberg, F. P. (2009). Point process diagnostics based on weighted second-order statistics and their asymptotic properties. Ann. Inst. Statist. Math. 61 929–948. MR2556772 10.1007/s10463-008-0177-1Adelfio, G. and Schoenberg, F. P. (2009). Point process diagnostics based on weighted second-order statistics and their asymptotic properties. Ann. Inst. Statist. Math. 61 929–948. MR2556772 10.1007/s10463-008-0177-1
Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automat. Control AC-19 716–723. MR423716 10.1109/TAC.1974.1100705Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automat. Control AC-19 716–723. MR423716 10.1109/TAC.1974.1100705
Baddeley, A. J., Møller, J. and Waagepetersen, R. (2000). Non- and semi-parametric estimation of interaction in inhomogeneous point patterns. Stat. Neerl. 54 329–350. MR1804002 10.1111/1467-9574.00144Baddeley, A. J., Møller, J. and Waagepetersen, R. (2000). Non- and semi-parametric estimation of interaction in inhomogeneous point patterns. Stat. Neerl. 54 329–350. MR1804002 10.1111/1467-9574.00144
Baddeley, A., Turner, R., Møller, J. and Hazelton, M. (2005). Residual analysis for spatial point processes. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 617–666. MR2210685 1112.62302 10.1111/j.1467-9868.2005.00519.xBaddeley, A., Turner, R., Møller, J. and Hazelton, M. (2005). Residual analysis for spatial point processes. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 617–666. MR2210685 1112.62302 10.1111/j.1467-9868.2005.00519.x
Clements, R. A., Schoenberg, F. P. and Veen, A. (2010). Evaluation of space–time point process models using super-thinning. UCLA Statistics Preprints 579 1–18.Clements, R. A., Schoenberg, F. P. and Veen, A. (2010). Evaluation of space–time point process models using super-thinning. UCLA Statistics Preprints 579 1–18.
Cressie, N. A. C. (1993). Statistics for Spatial Data. Wiley, New York. MR1239641 0799.62002Cressie, N. A. C. (1993). Statistics for Spatial Data. Wiley, New York. MR1239641 0799.62002
Field, E. H. (2007). Overview of the working group for the development of Regional Earthquake Likelihood Models (RELM). Seismological Research Letters 78 7–16.Field, E. H. (2007). Overview of the working group for the development of Regional Earthquake Likelihood Models (RELM). Seismological Research Letters 78 7–16.
Gerstenberger, M. C., Wiemer, S., Jones, L. M. and Reasenberg, P. A. (2005). Real-time forecasts of tomorrow’s earthquakes in California. Nature 435 328–331.Gerstenberger, M. C., Wiemer, S., Jones, L. M. and Reasenberg, P. A. (2005). Real-time forecasts of tomorrow’s earthquakes in California. Nature 435 328–331.
Guan, Y. (2009). On nonparametric variance estimation for second-order statistics of inhomogeneous spatial point processes with a known parametric intensity form. J. Amer. Statist. Assoc. 104 1482–1491. MR2750573 1205.62140 10.1198/jasa.2009.tm08541Guan, Y. (2009). On nonparametric variance estimation for second-order statistics of inhomogeneous spatial point processes with a known parametric intensity form. J. Amer. Statist. Assoc. 104 1482–1491. MR2750573 1205.62140 10.1198/jasa.2009.tm08541
Helmstetter, A., Kagan, Y. Y. and Jackson, D. D. (2007). High-resolution time-independent grid-based forecast M≥5 earthquakes in California. Seismological Research Letters 78 78–86.Helmstetter, A., Kagan, Y. Y. and Jackson, D. D. (2007). High-resolution time-independent grid-based forecast M≥5 earthquakes in California. Seismological Research Letters 78 78–86.
Kagan, Y. Y., Jackson, D. D. and Rong, Y. (2007). A testable five-year forecast of moderate and large earthquakes in southern California based on smoothed seismicity. Seismological Research Letters 78 94–98.Kagan, Y. Y., Jackson, D. D. and Rong, Y. (2007). A testable five-year forecast of moderate and large earthquakes in southern California based on smoothed seismicity. Seismological Research Letters 78 94–98.
Lewis, P. A. W. and Shedler, G. S. (1979). Simulation of nonhomogeneous Poisson processes by thinning. Naval Res. Logist. Quart. 26 403–413. MR546120 0497.60003 10.1002/nav.3800260304Lewis, P. A. W. and Shedler, G. S. (1979). Simulation of nonhomogeneous Poisson processes by thinning. Naval Res. Logist. Quart. 26 403–413. MR546120 0497.60003 10.1002/nav.3800260304
Meyer, P. (1971). Demonstration simplifiée d’un théorème de Knight. In Séminaire de Probabilités V. Lecture Notes in Math. 191 191–195. Springer, Berlin. MR380972Meyer, P. (1971). Demonstration simplifiée d’un théorème de Knight. In Séminaire de Probabilités V. Lecture Notes in Math. 191 191–195. Springer, Berlin. MR380972
Papangelou, F. (1972). Integrability of expected increments of point processes and a related random change of scale. Trans. Amer. Math. Soc. 165 483–506. MR314102 0236.60036 10.1090/S0002-9947-1972-0314102-9Papangelou, F. (1972). Integrability of expected increments of point processes and a related random change of scale. Trans. Amer. Math. Soc. 165 483–506. MR314102 0236.60036 10.1090/S0002-9947-1972-0314102-9
Schoenberg, F. (1999). Transforming spatial point processes into Poisson processes. Stochastic Process. Appl. 81 155–164. MR1694573 0962.60029 10.1016/S0304-4149(98)00098-2Schoenberg, F. (1999). Transforming spatial point processes into Poisson processes. Stochastic Process. Appl. 81 155–164. MR1694573 0962.60029 10.1016/S0304-4149(98)00098-2
Schoenberg, F. P. (2003). Multidimensional residual analysis of point process models for earthquake occurrences. J. Amer. Statist. Assoc. 98 789–795. MR2055487 10.1198/016214503000000710Schoenberg, F. P. (2003). Multidimensional residual analysis of point process models for earthquake occurrences. J. Amer. Statist. Assoc. 98 789–795. MR2055487 10.1198/016214503000000710
Schorlemmer, D., Gerstenberger, M. C., Wiemer, S., Jackson, D. D. and Rhoades, D. A. (2007). Earthquake likelihood model testing. Seismological Research Letters 78 17–27.Schorlemmer, D., Gerstenberger, M. C., Wiemer, S., Jackson, D. D. and Rhoades, D. A. (2007). Earthquake likelihood model testing. Seismological Research Letters 78 17–27.
Schorlemmer, D., Zechar, J. D., Werner, M. J., Field, E. H., Jackson, D. D., Jordan, T. H. and the RELM Working Group (2010). First results of the Regional Earthquake Likelihood Models Experiment. Pure and Applied Geophysics 167 859–876.Schorlemmer, D., Zechar, J. D., Werner, M. J., Field, E. H., Jackson, D. D., Jordan, T. H. and the RELM Working Group (2010). First results of the Regional Earthquake Likelihood Models Experiment. Pure and Applied Geophysics 167 859–876.
Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464. MR468014 0379.62005 10.1214/aos/1176344136 euclid.aos/1176344136
Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6 461–464. MR468014 0379.62005 10.1214/aos/1176344136 euclid.aos/1176344136
Shen, Z., Jackson, D. D. and Kagan, Y. Y. (2007). Implications of geodetic strain rate for future earthquakes, with a five-year forecast of M5 earthquakes in southern California. Seismological Research Letters 78 116–120.Shen, Z., Jackson, D. D. and Kagan, Y. Y. (2007). Implications of geodetic strain rate for future earthquakes, with a five-year forecast of M5 earthquakes in southern California. Seismological Research Letters 78 116–120.
Veen, A. and Schoenberg, F. P. (2005). Assessing spatial point process models using weighted K-functions: analysis of California earthquakes. In Case Studies in Spatial Point Process Models (A. Baddeley, P. Gregori, J. Mateu, R. Stoica and D. Stoyan, eds.) 293–306. Springer, New York. MR2232135 05243467 10.1007/0-387-31144-0_16Veen, A. and Schoenberg, F. P. (2005). Assessing spatial point process models using weighted K-functions: analysis of California earthquakes. In Case Studies in Spatial Point Process Models (A. Baddeley, P. Gregori, J. Mateu, R. Stoica and D. Stoyan, eds.) 293–306. Springer, New York. MR2232135 05243467 10.1007/0-387-31144-0_16
Wong, K. and Schoenberg, F. P. (2009). On mainshock focal mechanisms and the spatial distribution of aftershocks. Bull. Seismol. Soc. Amer. 99 3402–3412.Wong, K. and Schoenberg, F. P. (2009). On mainshock focal mechanisms and the spatial distribution of aftershocks. Bull. Seismol. Soc. Amer. 99 3402–3412.
Zhuang, J. (2006). Second-order residual analysis of spatiotemporal point processes and applications in model evaluation. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 635–653. MR2301012 1110.62128 10.1111/j.1467-9868.2006.00559.xZhuang, J. (2006). Second-order residual analysis of spatiotemporal point processes and applications in model evaluation. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 635–653. MR2301012 1110.62128 10.1111/j.1467-9868.2006.00559.x
Zhuang, J., Ogata, Y. and Vere-Jones, D. (2004). Analyzing earthquake clustering features by using stochastic reconstruction. Journal of Geophysical Research 109 B05301-17.Zhuang, J., Ogata, Y. and Vere-Jones, D. (2004). Analyzing earthquake clustering features by using stochastic reconstruction. Journal of Geophysical Research 109 B05301-17.