## For personal stuff, see the bradluen~! portal

### For my most recent thoughts on assessing earthquake forecasts, see the last talk I gave (Oct. 2008)

Luen, B. and Stark, P.B. (2008). Testing earthquake predictions. IMS Lecture Notes - Monograph Series. Probability and Statistics: Essays in Honor of David A. Freedman, 302-305. Institute for Mathematical Statistics Press, Beachwood, OH. Invited.

Statistical tests of earthquake predictions require a null hypothesis to model occasional chance successes. To define and quantify "chance success" is knotty. Some null hypothses ascribe chance to the Earth: seismicity is modelled as random. The null distribution of the number of successful prediction - or any other test statistics - is taken to be its distribution when the fixed set of predictions is applied to random seismicity. Such tests tacitly assume that the predictions do not depend on the observed seismicity. Conditioning on the predictions in this way sets a low hurdle for statistical significance. Consider this scheme: when an earthquake of magnitude 5.5 or greater occurs anywhere in the world, predict that another earthquake at least as large will occur within 21 days and an epicentral distance of 50 km. We apply this rule to the Harvard centroid-moment-tensor (CMT) catalogue for 2000-2004 to generate a set of predictions. The null hypothesis is that earthquake times are exchangeable on their magnitudes and locations and on the predictions. We generate random seismicity by permuting the times of events in the CMT catalogue. We consider an event successfully predicted only if (i) it falls in a prediction and (ii) there is no larger event within 50 km in the previous 21 days. The p-value for the observed success rate is less than 0.001: the method successfully predicts about 5% of earthquakes, far better than "chance", because the predictor exploits the clustering of earthquakes - occasional foreshocks - which the null hypothesis lacks. Rather than condition on the predictions and use a stochastic model for seismicity, it is preferable to treat the observed seismicity as fixed, and to compare the success rate of the predictions to the success rate of simple-minded predictions like those just described. If the proffered prediction do no better than a simple scheme, they have little value.