Journal of Geophysical Research: Solid Earth

2.1. Pattern Recognition (M8)

[6] In their pioneering work, Keilis‐Borok and Malinovskaya [1964] codified an observation of general increase in seismic activity depicted with the only measure, “pattern Sigma,” which is characterizing the trailing total sum of the source areas of the medium size earthquakes. Similar strict codifications of other seismicity patterns, such as a decrease of b value, an increase in the rate of activity, an anomalous number of aftershocks, etc, have been proposed later and define the M8 algorithm of earthquake forecast (see Keilis‐Borok and Kossobokov [1990a, 1990b] and Keilis‐Borok and Soloviev [2003] for useful reviews). In these algorithms, an alarm is defined when several precursory patterns are above a threshold calibrated in a training period. Predictions are updated periodically as new data become available. Most of the patterns used by this class of algorithms are reproduced by the model of triggered seismicity known as the ETAS model [see Sornette and Sornette, 1999; Helmstetter and Sornette, 2002a; Helmstetter et al., 2003]. The prediction gain G of the M8 algorithm, defined as the ratio between the fraction of predicted events over the fraction of time occupied by alarms, is usually in the range 3 to 10 (recall that a random predictor would give G = 1 by definition). A preliminary forward test of the algorithm for the time period July 1991 to June 1995 performed no better than the null hypothesis using a reshuffling of the alarm windows [Kossobokov et al., 1997]. Later tests indicated however a confidence level of 92% for the prediction of M7.5+ earthquakes by the algorithm M8‐MSc for real‐time intermediate‐term predictions in the Circum Pacific seismic belt, 1992–1997, and above 99% for the prediction of M ≥ 8 earthquakes [Kossobokov et al., 1999]. We use the term confidence level as 1 minus the probability of observing a predictability at least as good as what was actually observed, under the null hypothesis that everything is due to chance alone. As of July 2002, the scores (counted from the formal start of the global test initiated by this team since July 1991) are as follows: For M8.0+, 8 events occurred, 7 predicted by M8, 5 predicted by M8‐MSc; for M7.5+, 25 events occurred, 15 predicted by M8 and 7 predicted by M8‐MSc.

2.2. Short‐Term Forecast of Aftershocks

[7] Reasenberg and Jones [1989] and Wiemer [2000] have developed algorithms to predict the rate of aftershocks following major earthquakes. The rate of aftershocks of magnitude m following an earthquake of magnitude M is estimated by the expression

where b ≈ 1 is the b value of the Gutenberg‐Richter (GR) distribution. This approach neglects the contribution of seismicity prior to the main shock, and does not take into account the specific times, locations and magnitudes of the aftershocks that have already occurred. In addition, this model (1) assumes arbitrarily that the rate of aftershocks increases with the magnitude M of the main shock as ∼10^bM, which may not be correct. A careful measure of this scaling for the southern California seismicity gives a different scaling ∼10^αM with α = 0.8 [Helmstetter, 2003]. Moreover, an analytical study of the ETAS model [Sornette and Helmstetter, 2002] shows that, unless there is an upper cut‐off of the magnitude distribution, the case α ≥ b leads to an explosive seismicity rate, which is unrealistic to describe the seismic activity.

2.3. Branching Models

[8] Simulations using branching models as a tool for predicting earthquake occurrence over large time horizons were proposed in the work of Kagan [1973], and first implemented in Kagan and Knopoff [1977]. In a recent work, Kagan and Jackson [2000] use a variation of the ETAS model to estimate the rate of seismicity in the future but their predictions are only valid at very short times, when very few earthquakes have occurred between the present and the horizon.

[9] To solve this limitation and to extend the predictions further in time, Kagan and Jackson [2000] propose to use Monte Carlo simulations to generate many possible scenarios of the future seismic activity. However, they do not use this method in their forecasting procedure. These Monte Carlo simulations will be implemented in our tests, as we describe below. This method has already been tested by Vere‐Jones [1998] to predict a synthetic catalog generated using the ETAS model. Using a measure of the quality of seismicity forecasts in terms of mean information gain per unit time, he obtains scores usually worse than the Poisson method. We use below the same class of model and implement a procedure taking into account the cascade of triggering. We find, in contrast with the claim of Vere‐Jones [1998], a significant probability gain. We explain in section 5.6 the origin of this discrepancy, which can be attributed to the use of different measures for the quality of predictions.

[10] In the work of Helmstetter et al. [2003], the forecasting skills of algorithms based on three functions of the current and past seismicity (above a magnitude threshold) measured in a sliding window of 100 events have been compared. The functions are (1) the maximum magnitude M_max of the 100 events in that window, (2) the Gutenberg‐Richter b value measured on these 100 events by the standard Hill maximum likelihood estimator and (3) the seismicity rate λ defined as the inverse of the duration of the window. For each function, an alarm was declared for the target of an earthquake of magnitude larger than 6 when the function is either larger (for M_max and λ) or smaller (for b) than a threshold. These functions M_max, b and λ are similar and in some cases identical to precursors and predictors that have been studied by other authors. Helmstetter et al. [2003] found that these three predictors are considerably better than those obtained for a random prediction, with the prediction based on the seismicity rate λ being by far the best. This is a logical consequence of the model of interacting triggered seismicity used in Helmstetter et al. [2003] and also in the present work, in which any relevant physical observable is a function of the seismicity rate. At least in the class of interacting triggered seismicity, the largest possible amount of information is recovered by targeting the seismicity rate. All other targets are derived from it as linear or nonlinear transformations of it. Our present study refines and extends the preliminary tests of Helmstetter et al. [2003] by using a full model of seismicity rather than the coarse‐grained measure λ. We note also that the forecasting methods of Rundle et al. [2001, 2002] are based on a calculation of the coarse‐grained seismicity above a small magnitude threshold, which is then projected into the future.

3.1. Definition of the ETAS Model

[13] The ETAS model of triggered seismicity is defined as follows [Ogata, 1988]. We assume that a given event (the “mother”) of magnitude m_i ≥ m₀ occurring at time t_i and position gives birth to other events (“daughters”) in the time interval between t and t + dt and at point at the rate

[14] Φ(t) is the direct Omori law normalized to 1

where θ > 0, H(t) is the Heaviside function, and c is a regularizing timescale that ensures that the seismicity rate remains finite close to the main shock.

[15] Φ is a normalized spatial “jump” distribution from the mother to each of her daughter, which quantifies the probability for a daughter to be triggered at a distance from the mother, taking into account the spatial dependence of the stress induced by an earthquake.

[16] ρ(m) gives the total number of aftershocks triggered directly by an event of magnitude m

where m₀ is a lower bound magnitude below which no daughter is triggered. The adjective “direct” refers to the events of the first generation triggered in first‐order lineage from the mother event. The formulation of (3) and (4) is originally attributed to Utsu [1970].

[17] The model is complemented by the Gutenberg‐Richter (GR) law which states that each earthquake has a magnitude chosen according to the density distribution

P(m) is normalized: ∫_m0^∞P(m) dm = 1. When magnitudes are translated into energies, the GR law becomes the (power law) Pareto law. See Ogata [1989, 1992, 1999], Guo and Ogata [1997], and Helmstetter and Sornette [2002a] for a discussion of the values of the ETAS parameters c, α, θ, K and b in real seismicity. Note however that we have reasons to think that previous inversions of the ETAS parameters in real data are unreliable due to lack of concern for the unobserved seismicity below the completeness threshold. We are presently working on methods to address this question.

[18] In this first investigation, we limit ourselves to the time domain, studying time series of past seismicity summed over an overall spatial region, without taking into account information on earthquake locations. This amounts to integrating the local Omori law (2) over the whole space. In the following, we will thus use the integrated form of the local Omori law (2) given by

The complete model with both spatial and temporal dependence (2) has been studied by Helmstetter and Sornette [2002b] to derive the joint probability distribution of the times and locations of aftershocks including the whole cascade of secondary and later‐generations aftershocks. When integrating the rate of aftershocks calculated for the spatiotemporal ETAS model over the whole space, we recover the results used in this paper for the temporal ETAS model.

[19] We stress that not taking into account the spatial positions of the earthquakes is not saying that earthquakes occur at the same position. The synthetic catalogs we generate in space and time are similar to real catalogs and our procedure just neglects the information on space. Clearly, this is not what a full prediction method should do and it is clear that not using the information on the location of earthquakes will underestimate (sometimes grossly) the predictive skills that could be achieved with a full spatiotemporal treatment. However, the problem is sufficiently complex that we find it useful to go through this first step and develop the relevant concepts and first tests using only information on seismic time sequences.

3.2. Definition of the Average Branching Ratio n

[20] The key parameter of model (2) is the average number (or “branching ratio”) n of daughter‐earthquakes created per mother event. This average is performed over time and over all possible mother magnitudes. This average branching ratio n is a finite value for θ > 0 and for α < b, equal to

The normal regime corresponds to the subcritical case n < 1 for which the seismicity rate decays after a main shock to a constant level (in the case of a steady state source) with fluctuations in the seismic rate.

[21] Since n is defined as the average over all main shock magnitudes of the mean number of events triggered by a main shock, the branching ratio does not give the number of daughters of a given earthquake, because this number also depends on the specific value of its magnitude as shown by (4). As an example, take α = 0.8, b = 1, m₀ = 0 and n = 1. Then, a main shock of magnitude M = 7 will have on average 80000 direct aftershocks, compared to only 2000 direct aftershocks for an earthquake of magnitude M = 5 and less than 0.2 aftershocks for an earthquake of magnitude M = 0.

[22] The branching ratio defined by (7) is the key parameter of the ETAS model, which controls the different regimes of seismic activity. There are two observable interpretations for this parameter [Helmstetter and Sornette, 2003]. The branching ratio can be defined as the ratio of triggered events over total seismicity when looking at catalog of seismicity at large scale. The branching ratio is also equal to the ratio of the number of secondary and later‐generations aftershocks over the total number of aftershocks within a single aftershock sequence.

4.1. Formulation of the Global Seismicity Rate and Renormalized Omori's Law

[25] We define the “bare propagator” ϕ(t) of the seismicity as the integral of (2) over all magnitudes

which is normalized to n since Φ(t) is normalized to 1. The meaning of the adjective “bare” will become clear below, when we demonstrate that cascades of triggered events renormalize ϕ(t) into an effective (“renormalized” or “dressed”) propagator K(t). This terminology is borrowed from statistical and condensed‐matter physics which deal with physical phenomena occurring at multiple scales in which similar cascades of fluctuations lead to a renormalization of “bare” into “dressed” properties when going from small to large scales. See also Sornette and Sornette [1999] and Helmstetter and Sornette [2002a], where this terminology was introduced in the present context.

[26] The total seismicity rate λ(t) at time t is given by the sum of an “external” source s(t) and the aftershocks triggered by all previous events

where ϕ_mi(t − t_i) is defined by (2). Here, “external” source refers to the concept that s(t) is the rate of earthquakes not triggered by other previous earthquakes. This rate acts as a driving force ensuring that the seismicity does not vanish and models the effect of the external tectonic forcing.

[27] Taking the ensemble average of (9) over many possible realizations of the seismicity (or equivalently taking the mathematical expectation), we obtain the following equation for the first moment or statistical average N(t) of λ(t) [Sornette and Sornette, 1999; Helmstetter and Sornette, 2002a]

The average seismicity rate is the solution of this self‐consistent integral equation, which embodies the fact that each event may start a sequence of events which can themselves trigger secondary events and so on. The cumulative effect of all the possible branching paths of activity gives rise to the net seismic activity N(t). Expression (10) states that the seismic activity at time t is due to a possible external source s(t) plus the sum over all past times τ of the total previous activities N(τ) that may trigger an event at time t according to the bare Omori law ϕ(t − τ).

[28] The global rate of aftershocks including secondary and later‐generations aftershocks triggered by a main shock of magnitude M occurring at t = 0 is given by ρ(M) K(t)/n, where the renormalized Omori law K(t) is obtained as a solution of (10) with the general source term s(t) replaced by the Dirac function δ(t):

The solution for K(t) can be obtained as the following series [Helmstetter and Sornette, 2002a] The infinite sum expansion is valid for t > c, and t* is a characteristic time measuring the distance to the critical point n = 1 defined by t* is infinite for n = 1 and becomes very small for n ≪ 1. The leading behavior of K(t) at short times reads showing that the effect of the cascade of secondary aftershocks renormalizes the bare Omori law Φ(t) ∼ 1/t^1+θ given by (8) into K(t) ∼1/t^1−θ, as illustrated by Figure 1.

[29] Once the seismic response K(t) to a single event is known, the complete average seismicity rate N(t) triggered by an arbitrary source s(t) can be obtained using the theorem of Green functions for linear equations with source terms [Morse and Feshbach, 1953]

Expression (15) provides the general solution of (10).

4.2. Multiple Source Prediction Problem

[30] We assume that seismicity which occurred in the past until the “present” time u, and which does trigger future events, is observable. The seismic catalog consists of a list of entries {(t_i, m_i), t_i < u} giving the times t_i of occurrence of the earthquakes and their magnitude m_i. Our goal is to set up the best possible predictor for the seismicity rate for the future from time u to time t > u, based on the knowledge of this catalog {(t_i, m_i), t_i < u}. The time difference t − u is called the horizon. In the ETAS model studied here, magnitudes are determined independently of the seismic rate, according to the GR distribution. Therefore the sole meaningful target for prediction is the seismic rate. Once its forecast is issued, the prediction of strong earthquakes is obtained by combining the GR law with the forecasted seismic rate. The average seismicity rate N(t) at time t > u in the future is made of two contributions:

[31] 1. The external source of seismicity of intensity μ at time t plus the external source of seismicity that occurred between u and t and their following aftershocks that may trigger an event at time t.

[32] 2. The earthquakes that have occurred in the past at times t_i < u and all the events they triggered between u and t and their following aftershocks that may trigger an event at time t. We now examine each contribution in turn.

5.1. Fixed Present and Variable Forecast Horizon

[53] Figure 3 illustrates the problem of forecasting the aftershock seismicity following a large M = 7 event. Imagine that we have just witnessed the M = 7 event and want to forecast the seismic activity afterward over a varying horizon from days to years in the future. In this simulation, u is kept fixed at the time just after the M = 7 event and t is varied. A realization of the instantaneous rate of seismic activity (number of events per day) of a synthetic catalog is shown by the black dots. This simulation has been performed with the parameters n = 0.8, α = 0.8, b = 1, c = 0.001 day, m₀ = 3 and μ = 1 event per day. This single realization is compared with two forecasting algorithms: the sum of the bare propagators of all past events t_i ≤ u, and the median of the seismicity rate obtained over 500 scenarios generated with the ETAS model, using the same parameters as used for generating the synthetic catalog we want to forecast, and taking into account the specific realization of events in each scenario up to the present. Figure 4 is the same as Figure 3 but shows the seismic activity as a function of the logarithm of the time after the main shock. These two figures illustrate clearly the importance of taking into account all the cascades of still unobserved triggered events in order to forecast correctly the future rate of seismicity beyond a few minutes. The aftershock activity forecast gives a very reasonable estimation of the future activity rate, while the extrapolation of the bare Omori law of the strong M = 7 event underestimates the future seismicity beyond half‐an‐hour after the strong event.

5.2. Varying “Present” With Fixed Forecast Horizon

[54] Figure 5 compares a single realization of the seismicity rate observed and summed over a 5 days period and divided by 5 so that it is expressed as a daily rate, with the predicted seismicity rate using either the sum of the bare propagators of the past seismicity or using the median of 100 scenarios generated with the same parameters as for the synthetic catalog we want to forecast: n = 0.8, c = 0.001 day, μ = 1 event per day, m₀ = 3, b = 1 and α = 0.8. The forecasting methods calculate the total number of events over each 5 day period lying ahead of the present, taking into account all past seismicity including the still unobserved triggered seismicity. This total number of forecasted events is again divided by 5 to express the prediction as daily rates. The thin solid lines indicate the first and 9th deciles of the distributions of the number of events observed in the pool of 100 scenarios. Stars indicate the occurrence of large M ≥ 7 earthquakes. Only a small part of the whole time period used for the forecast is shown, including the largest M = 8.5 earthquake of the catalog, in order to illustrate the difference between the realized seismicity rate and the different methods of forecasting.

[55] The realized seismicity rate is always larger than the seismicity rate predicted using the sum of the bare propagators of the past activity. This is because the seismicity that will occur up to 5 days in the future is dominated by events triggered by earthquakes that will occur in the next 5 days. These events are not taken into account by the sums of the bare propagators of the past seismicity. The realized seismicity rate is close to the median of the scenarios (crosses), and the fluctuations of the realized seismicity rate are in good agreement with the expected fluctuations measured by the deciles of the distributions of the seismicity rate over all scenarios generated.

[56] Figure 6 compares the predictions of the seismicity rate over a 5 day horizon with the seismicity of a typical synthetic catalog; a small fraction of the history was shown in Figure 5. This comparison is performed by plotting the predicted number of events in each 5 day horizon window as a function of the actual number of events. The open circles (crosses) correspond to the forecasts using the median of 100 scenarios (the sum of the bare Omori propagators of the past seismicity). This figure uses a synthetic catalog of N = 200000 events of magnitude larger than m₀ = 3 covering a time period of 150 years. The dashed line corresponds to the perfect prediction when the predicted seismicity rate is equal to the realized seismicity rate. This figure shows that the best predictions are obtained using the median of the scenarios rather than using the bare propagator, which always underestimates the realized seismicity rate, as we have already shown.

[57] The most striking feature of Figure 6 is the existence of several clusters, reflecting two mechanisms underlying the realized seismicity: (1) cluster LL with large predicted seismicity and large realized seismicity; (2) cluster SL with small predicted seismicity and large realized seismicity; (3) cluster SS with small predicted seismicity and small realized seismicity. Cluster LL along the diagonal reflects the predictive skill of the triggered seismicity algorithm: this is when future seismicity is triggered by past seismicity. Cluster SL lies horizontally at low predicted seismicity rates and shows that large seismicity rates can also be triggered by an unforecasted strong earthquake, which may occur even when the seismicity rate is low. This expresses a fundamental limit of predictability since the ETAS model permits large events even for low prior seismicity, as the earthquake magnitudes are drawn from the GR independently of the seismic rate. About 20% of the large values of the realized seismicity rate above 10 events per day fall in the LL cluster, corresponding to a predictability of about 20% of the large peaks of realized seismic activity. The cluster SS is consistent with a predictive skill but small seismicity is not usually an interesting target. Note that there is no cluster of large predicted seismicity associated with small realized seismicity.

[58] Figure 7 is the same as Figure 5 for a longer time window of 50 days, which stresses the importance of taking into account the yet unobserved future seismicity in order to accurately forecast the level of future seismicity. Figure 8 is the same as Figure 6 for the forecast time window of 50 days with forecasts updated each 50 days. Increasing the time window T of the forecasts from 5 to 50 days leads to a smaller variability of the predicted seismicity rate. However, fluctuations of the seismicity rate of one order of magnitude can still be predicted with this model. The ETAS model therefore performs much better than a Poisson process for large horizons of 50 days.

5.3. Error Diagrams and Prediction Gains

[59] In order to quantify the predictive skills of different prediction algorithms for the seismicity of the next five days, we use the error diagram [Molchan, 1991, 1997; Molchan and Kagan, 1992]. The predictions are made from the present to 5 days in the future and are updated each 0.5 day. Using a shorter time between each prediction, or updating the prediction after each major earthquake, will obviously improve the predictions, because large aftershocks occur often just after the main shock. However, in practice the forecasting procedure is limited by the time needed to estimate the location and magnitude of an earthquake. Moreover, predictions made for very short times in advance (a few minutes) are not very useful.

[60] An error diagram requires the definition of a target, for instance M ≥ 6 earthquakes, and plots the fraction of targets that were not predicted as a function of the fraction of time occupied by the alarms (total duration of the alarms normalized by the duration of the catalog). We define an alarm when the predicted seismic rate is above a threshold. Recall that in our model the seismic rate is the physical quantity that embodies completely all the available information on past events. All targets one might be interested in derive from the seismic rate.

[61] Figure 9 presents the error diagram for M ≥ 6 targets, using a time window T = 5 days to estimate the seismicity rate, and a time dT = 0.5 days between two updates of the predictions. We use different prediction algorithms, either the bare propagator (dots), the median (circles) or the mean (triangles) number of events obtained for the 100 scenarios already generated to obtain Figures 5 and 6. Each point of each curve corresponds to a different threshold ranging from 0.1 to 1000 events per day. The results for these three prediction algorithms are considerably better than those obtained for a random prediction, shown as a dashed line for reference.

[62] Ideally, one would like the minimum numbers of failures and the smallest possible alarm duration. Hence a perfect prediction corresponds to points close to the origin. In practice, the fraction of failure to predict is 100% without alarms and the gain of the prediction algorithm is quantified by how fast the fraction of failure to predict decreases from 100% as the fraction of alarm duration increases. Formally, the gain G reported below is defined as the ratio of the fraction of predicted targets (=1 – number of failures to predict) divided by the fraction of time occupied by alarms. A completely random prediction corresponds to G = 1.

[63] We observe that about 50% of the M ≥ 6 earthquakes can be predicted with a small fraction of alarm duration of about 20%, leading to a gain of 2.5 for this value of the alarm duration. The gain is significantly stronger for shorter fractions of alarm duration: as shown in Figure 9b, 25% of the M ≥ 6 earthquakes can be predicted with a small fraction of alarm duration of about 2%, leading to a gain of 12.5. The origin of this good performance for only a fraction of the targets has been discussed in relation with Figure 6, and is associated with those events that occur in times of large seismic rate (cluster along the diagonal in Figure 6). Figure 9c shows the dependence of the prediction gain G as a function of the alarm duration: the three prediction schemes give approximately the same power law increase with an exponent close to 1/2 of the gain as the duration of alarm decreases. For small alarm duration, the gain reaches values of several hundreds. The saturation at very small values of the alarm duration is due to the effect that only a few targets are sampled. Figures 10 and 11 are similar to Figure 9, respectively for a smaller target of magnitudes larger than 5 and a larger target of magnitudes larger than 7.

[64] Table 1 presents the results for the prediction gain and for the number of successes using different choices of the time window T and of the update time dT between two predictions, and for different values of the target magnitude between 5 and 7. The prediction gain decreases if the time between two updates of the prediction increases, because most large earthquakes occur at very short times after a previous large earthquake. In contrast, the prediction gains do not depend on the time window T for the same value of the update time dT.

[65] The prediction gain is observed to increase significantly with the target magnitude, especially in the range of small fraction of alarm durations (see Table 1 and Figures 9, Figures 9–11). However, this increase of the prediction gain does not mean that large earthquakes are more predictable than smaller ones, in contrast with, for example, the critical earthquake theory [Sornette and Sammis, 1995; Jaumé and Sykes, 1999; Sammis and Sornette, 2002]. In the ETAS model, the increase of the prediction gain with the target magnitude is due to the decrease of the number of target events with the target magnitude. Indeed, choosing N events at random in the catalog independently of their magnitude gives on average the same prediction gain as for the N largest events. This demonstrates that the larger predictability of large earthquakes is solely a size effect.

[66] We now clarify the statistical origin of this size effect. Let us consider a catalog of total duration D with a total number N of events analyzed with D/T time windows with horizon T. These D/T windows can be sorted by decreasing seismicity r₁ > r₂ >. > r_n >., where r_i is the i‐th largest number of events in a window of size T. There are n₁, n₂,., n_i,. windows of type 1, 2,., i,. respectively, such that Σ_ir_in_i = N. Then, the frequency‐probability that an earthquake drawn at random from the catalog falls within a window of type i is

We have found that, over more than three decades spanning from 1 event to 10³–10⁴ events per window, the cumulative distribution of these p_i's is a power law with exponent approximately equal to κ = 0.4. This power law is found for the observable realized seismicity as well as for the seismicity predicted by the different methods discussed above. Such a small exponent κ implies that the few windows that happen to have the largest number of events contain a significant fraction of the total seismicity. It can be shown [Feller, 1971] that, in absence of any constraint, the single window with the largest number of events contains on average 1 − κ = 60% of the total seismicity. This would imply that there is a 60% probability that an earthquake drawn at random within the catalog (of 150 years) belongs to this single window of 5 days. This effect is moderated because the random variables p_i have to sum up to 1 by definition. This can be shown to entail a roll‐off of the cumulative distribution depleting the largest values of p_i. Empirically, we find that the most active window out of the approximately 54,000 daily windows of our 150 years long catalog contains only 3% of the total number of events. While this value of 3% is smaller than the prediction of 60% in absence of normalization, it is considerably larger than the “democratic” result which would predict a fraction of about 0.002% of the seismicity in each window. Since a high seismicity rate implies strong interactions and triggering between earthquakes and is usually associated with recent past high seismicity; the events in such a window are highly predictable. When the number of targets increases, one starts to sample statistically other windows with smaller seismicity which have a weaker relation with triggered seismicity and thus present less predictive power.

[67] In our previous discussion, we have not distinguished the skills of the three algorithms, because they perform essentially identically with respect to the assigned targets. This is very surprising from the perspective offered by all our previous analysis showing that the naive use of the direct Omori law without taking into account the effect of any indirect triggered seismicity strongly underestimate the future seismicity. We should thus expect a priori that this prediction scheme should be significantly worse than the two others based on a correct counting of all unobserved triggered seismicity. The explanation for this paradox is given by examining Figure 12, which presents further insight in the prediction methods applied to the synthetic catalogs used in Figures 3-Figures 3–11. Figure 12 shows three quantities as a function of the threshold in seismicity rate used to define an alarm, for each of the three algorithms. These quantities are respectively the duration of alarms normalized by the total duration of the catalog shown in Figure 12a, the fraction of successes (=1 – failure to predict) shown in Figure 12b and the prediction gain shown in Figure 12c. These three figures tell us that the incorrect level of seismic activity predicted by the bare Omori law approach can be compensated by the use of a lower alarm threshold. In other words, even if the seismicity rate predicted by the bare Omori law approach is wrong in absolute values, its time evolution in relative terms contains basically the same information as the full‐fledged method taking into account all unobserved triggered seismicity. Therefore an algorithm that can detect a relative change of seismicity can perform as well as the complete approach for the forecast of the assigned targets. This is an illustration of the fact that predictions of different targets can have very different skills which depend on the targets. Using the full‐fledged renormalized approach is the correct and only method to get the best possible predictor of future seismicity rate. However, other simpler and more naive methods can perform almost as well for more restricted targets, such as the prediction of only strong earthquakes.

5.4. Optimization of Earthquake Prediction

[68] The optimization of the prediction method requires the definition of a loss function γ, which should be minimized in order to determine the optimum alarm threshold of the prediction method. The probability gain defined in the previous section cannot be used as an optimization method, because it is maximum for zero alarm time. The strategies that optimize the probability gain are thus very impractical. An error function commonly used [e.g., Molchan and Kagan, 1992] is the sum of the two types of errors in earthquake prediction, the fraction of alarm duration τ and the fraction of missed events v. This loss function is illustrated in Figure 13 for the same numerical simulation of the ETAS model as in the previous section, using a prediction time window of 5 days and an update time of 0.5 day, for different values of the target magnitude between 5 and 7. For a prediction algorithm that has no predictive skill, the loss function γ should be close to 1, independently of the alarm duration τ. This is indeed what we observe for the prediction algorithm using a Poisson process, with a constant seismicity rate equal to the average seismicity rate of the realized simulation. For the prediction methods based on the ETAS model, we obtain a significant predictability by comparison with the Poisson process (Figure 13). The loss function is minimum for a fraction of alarm duration of about 10%, and decreases with the target magnitude M_t from 0.9 for M_t = 5 down to 0.7 for M_t = 7. We expect that the minimum loss function will be much lower when using the information on earthquake locations.

5.5. Influence of the ETAS Parameters on the Predictability

[69] We now test the influence of the model parameters α, n, m₀ and p on the predictability. We did not test the influence of the b value, because this parameter is rather well constrained in seismicity, and because its influence is felt only relative to α. We keep c equal to 0.001 day because this parameter is not critical as long as it is small. The external loading μ is also fixed to 1 event/day because it is only a multiplicative factor of the global seismicity. The value of the minimum magnitude m₀ above which earthquakes may trigger aftershocks of magnitude larger than m₀ is very poorly constrained. This value is no larger than 3 for the Southern California seismicity, because there are direct evidences of M3+ earthquakes triggered by M = 3 earthquakes [Helmstetter, 2003]. We are limited in our exploration of small values of m₀ because the number of earthquakes increases rapidly if m₀ decreases, and thus the computation time becomes prohibitive. We have tested the values m₀ = 1 and m₀ = 2 and find that the effect of a decrease of m₀ is simply to multiply the seismicity rate obtained for M ≥ 3 by a constant factor. Using a smaller m₀ does not change the temporal distribution of seismicity and therefore does not change the predictability of the system.

[70] We use the minimum value of the error function γ introduced in the previous section to characterize the predictability of the system. This function is estimated from the seismicity rate predicted using the bare propagator, as done in previous studies [Kagan and Knopoff, 1987; Kagan and Jackson, 2000]. While all three prediction methods that we have investigated give approximately the same results for the error diagram, the method of the bare propagator is much faster, which justifies to use it. The results are summarized in Table 2, which gives the minimum value of the error function γ for each set of parameters, and the corresponding values of the alarm duration, the proportion of predicted M ≥ 6 events and the prediction gain. All values of γ are in the range 0.6–0.9, corresponding to a small but significant predictability of the ETAS model. The predictability increases (i.e., γ decreases) with α, n and p, because for large α, large n and/or large p, there are larger fluctuations of the seismicity rate which have stronger impacts for the future seismicity. The minimum magnitude m₀ has no significant influence on the predictability. We recover the same pattern when using another estimate of the predictability measured by the prediction gain G_1% for an alarm duration of 1%.

5.6. Information Gain

[71] We now follow Kagan and Knopoff [1977], who introduced the entropy/information concept linking the likelihood gain to the entropy per event and hence to the predictive power of the fitted model, and Vere‐Jones [1998], who suggested using the information gain to compare different models and to estimate the predictability of a process.

[72] For a Poisson process, and assuming a constant magnitude distribution given by (5), the probability p_i to have at least one event above the target magnitude M_t in the time interval (t_i, t_i + T) can be evaluated from the average seismicity rate λ_i above m₀ by

Expression (29) should not be used for the ETAS model, because events occurring between t_i and t_i + T are not independent. Our forecasting algorithm provides the number N_j of events larger than m₀ occurring in the time interval (t_i, t_i + T) for each scenario j = 1, 2, …, 100. We can estimate the probability of having at least one event larger than M_t from the number N_j of events in each scenario, assuming a constant magnitude distribution given by (5), using the expression The bracket expresses an averaging over all the scenarios. Notwithstanding its improvement over (29), expression (30) is still nonexact because it neglects the correlations between the sequence of magnitudes and the seismicity rate. While the magnitude of each event is drawn independently from the magnitude distribution (5), scenarios that have a larger number of events in the time window (t_i, t_i + T) have on average a larger proportion of large events than scenarios with a smaller number of events, because a large earthquake triggers a large increase of the seismicity rate. Figure 14 shows the probability p_i given by (30) obtained for different choices of the target magnitude M_t for the same sequence as in Figure 5.

[73] The binomial score B compares the prediction p_i with the realization X_i, with X_i = 1 if a target event occurred in the interval (t_i, t_i + T) and X_i = 0 otherwise. For the whole sequence of intervals (t_i, t_i + T), the binomial score is defined by

where the sum is performed over all (nonoverlapping) time windows covering the whole duration of the catalog. The first term represents the contribution to the score from those intervals which contain an event, and the second term the contribution to the score from those intervals which contain no event.

[74] In order to test the performance of our forecasting algorithm, we compare the binomial score B_ETAS of the ETAS model with two time‐independent processes. First, we use a Poisson process with a seismicity rate equal to the average seismicity rate of the realized catalog, as in [Vere‐Jones, 1998], and use equation (29) to estimate the probability p_i. Because the Poisson process assumes a uniform temporal distribution of target events and thus neglects clustering of large events, it over‐estimates the proportion of intervals which have at least one target event. Indeed, the probability of having several events of magnitude M ≥ M_t in the same time window is much higher for the ETAS model than for a Poisson process. We thus propose another null hypothesis. We define a time‐independent non‐Poissonian process by putting all values of p_i equal to the fraction of intervals in the realized catalog that have at least a target events. For small time intervals and/or large target magnitudes, the proportion of intervals that have several target events is very small, and the two time‐independent processes give similar results. Another alternative to construct a time‐independent null‐hypothesis including clustering is to replace the Poisson distribution assumed in (29) by a negative binomial law [Kagan and Jackson, 2000], which has a larger variance.

[75] The results for different choices of the time interval T and of the target magnitude M_t are listed in Table 3. We evaluate the binomial score B for different prediction algorithms: (1) the ETAS model using numerical simulations to evaluate the probability p_i given by (30) (B_ETAS), (2) using the sum of the bare propagators of the past seismicity to estimate the average seismicity rate λ_i and assuming a Poisson process (29) to estimate the probability p_i from the rate λ_i (B_ϕ), (3) a Poisson process (B_pois) and (iv) the second null‐hypothesis, with p_i fixed to the observed fraction of time intervals which have at least a target event (B_nh). The results for the forecasting method based on simulations of the ETAS model (B_ETAS) are generally better than for the time‐independent processes, i.e., the binomial score for the ETAS model is larger than the score obtained with a Poisson process (B_pois) or with the second null‐hypothesis (B_nh). The second null hypothesis (B_nh) performs much better than a Poisson process because it takes into account clustering. For large time scales T and small target magnitude M_t, it performs even better than predictions based on the ETAS model. The score B_ETAS, which takes into account the cascade of secondary aftershocks, is significantly better than the score B_ϕ obtained with the bare propagator, even for short time intervals T, because this predictor under‐estimates the realized seismicity rate. For large time intervals T ≥ 10 days, and for large target magnitudes, the scores B_ϕ for the bare propagator are even worse than the results obtained with a Poisson process. Our results are in disagreement with those reported in Vere‐Jones [1998] on the same ETAS model: we conclude that the ETAS model has a significantly higher predictive power than the Poisson process while Vere‐Jones [1998] concludes that the forecasting performance of the ETAS model is worse than with the Poisson model. Vere‐Jones and Zhuang's procedure and ours are very similar. They use the same method to generate ETAS simulations and to update the predictions at rigid time intervals, with a similar time between two updates of the predictions. They use the same method to estimate the probability of having a target event for the Poisson process. Rather than using equation (30) to derive the probability p_i from the number of events in each scenario as done in this work, they measure directly this probability from the fraction of scenarios which have at least a target event. We have compared the two methods and found that the method of Vere‐Jones gives results very similar to ours. However, for large target events and small time intervals, the method of Vere‐Jones requires to generate a huge number of scenarios to obtain accurate estimates of the fraction of scenarios which have at least a target event. We thus believe that our method might be better in this case and give a more accurate estimate of the probability p_i of having at least a target event. This is one possible origin of the discrepancy between Vere‐Jones' results and ours.

[76] The ETAS parameters used in the two studies are also very similar and cannot account for the disagreement. The tests of Vere‐Jones [1998] have been performed using a α/b ratio of 0.57/1.14 = 0.5 smaller than the value α/b = 0.8 used in our simulations. This difference may lead to a smaller predictability for the simulations of Vere‐Jones [1998] because there are fewer large aftershock sequences. The branching ratio n = 0.78 used by Vere‐Jones [1998] is very close to our value n = 0.8. However, these difference in the ETAS parameters cannot explain why Vere‐Jones [1998] obtains a better predictability for a Poisson process than for the ETAS model. Vere‐Jones [1998] concludes that the Poisson process is better predictive than ETAS because the binomial score measured for time periods containing target events, i.e., by taking only the first term of equation (30), is larger for the Poisson process than for ETAS. While we agree with this point, we note that the fact that the first term of the binomial score is larger for the Poisson process than for ETAS does not imply that the Poisson process is more predictive. Indeed, the score for periods containing events is maximum for a simple model specifying a probability p_i = 1 of having an event for all time intervals. Because this simple model does not miss any event, it gives the maximum binomial score for periods containing target events, but is of course not better than the ETAS or the Poisson model if we take into account the second term of the binomial score corresponding to periods without target events. If we take our second definition of the Poisson process, taking into account clustering, we obtain a better score with the ETAS model for periods containing target events than for the Poisson process. We thus think that the conclusion of Vere‐Jones [1998] that the ETAS model is sometimes less predictive than the Poisson process is due to an inadequate measure of the predictability. We thus caution that a suitable assessment of the forecasting skills of a model requires several complementary quantitative measures, such as the predicted versus realized seismicity rates, the error diagram and predictability gain and the entropy‐information gains, and a large number of target events.