Predictability of the Arctic sea ice edge

Skillful sea ice forecasts from days to years ahead are becoming increasingly important for the operation and planning of human activities in the Arctic. Here we analyze the potential predictability of the Arctic sea ice edge in six climate models. We introduce the integrated ice‐edge error (IIEE), a user‐relevant verification metric defined as the area where the forecast and the “truth” disagree on the ice concentration being above or below 15%. The IIEE lends itself to decomposition into an absolute extent error, corresponding to the common sea ice extent error, and a misplacement error. We find that the often‐neglected misplacement error makes up more than half of the climatological IIEE. In idealized forecast ensembles initialized on 1 July, the IIEE grows faster than the absolute extent error. This means that the Arctic sea ice edge is less predictable than sea ice extent, particularly in September, with implications for the potential skill of end‐user relevant forecasts.


Introduction
The ongoing retreat of sea ice opens new opportunities for human activities in the Arctic [Emmerson and Lahn, 2012].The volume of marine transport through the Arctic, for example, could increase rapidly in the coming decades [Smith and Stephenson, 2013].For the anticipated developments to be sustainable, reliable forecasts of Arctic weather and climate, including sea ice in particular, are indispensable, especially in order to reduce risks of human and environmental emergencies and to enable efficient responses.Numerical weather prediction systems are also expected to benefit from the inclusion of skillful sea ice forecast components, in polar regions [e.g., Pellerin et al., 2004] and beyond [e.g., Balmaseda et al., 2010;Jung et al., 2014].
Existing Arctic sea ice forecasting systems based on global or regional coupled (atmosphere-sea ice-ocean) general circulation models are starting to be used operationally for short-term (up to ∼10 days ahead) sea ice predictions [Smith et al., 2015a].Seasonal forecasts based on similar dynamical or simpler statistical models are increasingly being produced, and most of these forecasts are evaluated in the Sea Ice Prediction Network (SIPN) Sea Ice Outlook [Stroeve et al., 2014].Despite these important recent developments, sea ice forecasts and their verification are still at an early stage.
There is no consensus yet on how best to assess the quality of sea ice forecasts, in particular when it comes to evaluating the spatial distribution of sea ice instead of just the pan-Arctic extent or volume (for an overview of current practice in sea ice verification, see Smith et al. [2015b]).Given the importance of the ice-edge position, the mean distance between the forecasted and the true (observed) ice edge has been used as a verification metric by several groups [Melsom et al., 2011;Posey et al., 2015;Dukhovskoy et al., 2015].This metric is both intuitive and relevant for potential sea ice forecast users but has two disadvantages: (i) to determine correct weights for the discrete segments of a contour not aligned with the model grid, which is necessary to compute this metric, is nontrivial, and (ii) the metric is overly sensitive to isolated patches of sea ice occurring in either the forecasted or the true (observed) sea ice distribution.The metric we are suggesting here does not have these drawbacks.
To assess whether forecast improvements are in principle possible, the inherent limits of predictability have been investigated in a number of studies based on individual models [Koenigk and Mikolajewicz, 2009;Blanchard-Wrigglesworth et al., 2011a;Holland et al., 2011;Tietsche et al., 2013;Day et al., 2014a]  project [Tietsche et al., 2014;Day et al., 2015;Hawkins et al., 2015].Under the assumption that the predictability of Arctic sea ice in the real world is captured by the set of models used, these "perfect-model" studies reveal that pan-Arctic sea ice extent and volume could be predicted much more skillfully than is currently possible with existing prediction systems [Stroeve et al., 2014;Blanchard-Wrigglesworth et al., 2015].This suggests that large model and/or initialization errors hamper current sea ice forecasting capabilities [see also Day et al., 2014b].For a recent review on Arctic sea ice predictability, see Guemas et al. [2014].
This paper has two main aspects: We (i) introduce an error metric for sea ice forecasts with a number of advantageous properties and (ii) use this metric to investigate the potential predictability of the Arctic ice edge in the APPOSITE simulations.We first recap the experimental setup and specifics of the model contributions in section 2, followed by a description of the integrated ice-edge error in section 3. We then analyze climatological variations of the Arctic ice edge in section 4, which is required to assess the error growth in the idealized forecasts discussed in section 5 in terms of predictable fractions.The paper closes with a summary and conclusions in section 6.

Model Data
This study is based on climate model data generated within the Arctic Predictability and Prediction On Seasonal to Interannual Timescales (APPOSITE) project.The contributing modeling groups conducted control simulations under constant "present-day" external forcing (1990 or 2005 conditions for ∼200 years, depending on the model), with one exception (see below).Idealized forecast ensembles were started from the control simulations on multiple dates, with the initial sea surface temperatures perturbed by white noise of amplitude 10 −4 K. Owing to the chaotic nature of the atmosphere, these minute random perturbations suffice to alter the atmospheric states of different ensemble members substantially within days, driving differences also in the ocean and sea ice.For details of the APPOSITE experimental setup and model data contributions, we refer to Day et al. [2015].
Here we use only those idealized forecasts initialized on 1 July for a number of individual years, allowing us to investigate the short-term predictability of the late-summer sea ice edge position.We limit the analysis to those models for which daily sea ice data were provided for the control simulation and for the idealized forecasts.Between the models, the number of start dates varies between 8 and 32, and the ensemble size varies between 8 and 16 (Table 1).CanCM4 is an exception in that the control simulation is conducted under transient 1970-2014 forcing.

The Integrated Ice-Edge Error
We distinguish between a forecasted and a true ice edge, keeping in mind that in real-world applications the true state can only be approximated based on observations.We define the integrated ice-edge error (IIEE) as the area where the forecast and the truth disagree on the ice concentration being above or below 15%, that is, the sum of all areas where the local sea ice extent is overestimated (O) or underestimated (U): and where A is the area of interest, here the Northern Hemisphere, c = 1 where the sea ice concentration is above 15% and c = 0 elsewhere, and subscripts f and t denote the forecast and the truth (Figure 1).The definition of the IIEE is equivalent to the so-called symmetric difference between the areas enclosed by the forecasted and the true ice edge.
The IIEE has a number of properties that make it a useful verification metric.(i) The IIEE is conceptually simple and straightforward to derive from modeled and observed gridded sea ice concentration data.(ii) Remote sensing data reveal the approximate ice-edge position for the past ∼35 years, enabling evaluation of sea ice forecasting systems with retrospective forecasts.(iii) The ice-edge position is an important characteristic of the sea ice cover and, accordingly, the IIEE much more relevant to potential forecast users than just the sea

Geophysical Research Letters
10.1002/2015GL067232 ice extent [see also Tietsche et al., 2014].(iv) The IIEE lends itself to decomposition into absolute extent error (AEE) and misplacement error (ME) components: and, as follows with equation ( 1), Note that the absolute extent error as defined above equals the absolute difference in the common sea ice extent between the forecasted and true states.
This decomposition highlights the limitations of sea ice extent verifications: the information contained in the misplacement error, arguably as important for forecast users because it reflects too much sea ice in one place and too little in another, is completely neglected.In fact, as we demonstrate below, the misplacement error typically dominates the pan-Arctic IIEE, in particular at short lead times.

Climatological Variability
To assess the potential predictability of the sea ice edge in terms of a predictable fraction, it is first required to quantify the forecast horizon given by the climatological amplitude of sea ice edge fluctuations.We define the climatological IIEE as follows: where l is the lag in years, and y 1 and y L are the first and last years of the finite period for which the climatological IIEE is computed.The term in parentheses is the same as the IIEE in equation ( 1) but with two states of the same realization (here a model control simulation), lagged by l years, interpreted as forecast and truth (a corresponding subscript (f,t) has been omitted in equations ( 1)-( 6) for simplicity).Note that due to the presence of a seasonal cycle, the climatological IIEE is a function of the time of the year.
In a stationary climate, IIEE clim as defined in equation ( 7) converges for l → ∞ to an actual IIEE clim where variability on all timescales is included.Given the finite simulation lengths considered here, we choose l = 10 which allows to retain a sufficiently large sample size N = y L − y 1 − l + 1 to estimate IIEE clim while minimizing potential spurious effects from trends seen in some of the simulations (due to transient forcing in CanCM4 and due to remaining drift in the course of equilibration in some of the other models).Indeed, we find that IIEE clim tends to increase slightly with l still beyond l = 10 (not shown), though this can also be partly due to low-frequency variability.Our analysis thus accounts for variability and predictability at timescales up to decadal only.In the following we refer to IIEE clim 10 as the climatological IIEE (IIEE clim ).The pan-Arctic climatological IIEE, later used to define the limit of predictability, varies between the models and as a function of the time of the year in the range (0.6-2.2) ⋅ 10 6 km 2 (Figure 2, left).There is a distinct offset between the models, with MIROC5 exhibiting the lowest and HadGEM1.2 and CanCM4 exhibiting the highest values, but the seasonal cycle is largely coherent between the models.Two main factors cause the variations in IIEE clim , namely, variations in (i) the climatological ice-edge length and (ii) the amplitude of lateral ice-edge variability (IIEE normalized by ice-edge length).A more detailed analysis for AWI-CM (not shown) reveals that the seasonal variations at least in this model are largely determined by the length of the ice edge.The amplitude of lateral ice-edge variability is approximately constant over the year, except for increased variability in late summer and early autumn, i.e., around the annual Arctic sea ice minimum.
The AEE contributes between 22% and 53% to the climatological IIEE, depending on the model and the time of the year (Figure 2, right).This implies that the Arctic sea ice extent alone reflects only one quarter to one half of the climatological variations of the sea ice edge.
To interpret the AEE/IIEE ratio, it is useful to think of ice-edge variations in a simplified way where the ice edge is composed of a fixed number of segments that are subject to fluctuations from independent identical random  7)) as a function of the time of the year for all models.(right) The fractional contribution of the AEE to IIEE clim (black scale on the left) and the corresponding degrees of freedom (DOF; red scale on the right, see equation ( 8)).
processes (e.g., autoregressive; see Text S1 and Figure S2 in the supporting information).In this simple model the expectation value for the fraction the AEE contributes to the IIEE is given by where DOF is the degrees of freedom, i.e., the number of independent segments.This simple relationship, which corresponds to the relation between the standard deviation of a sample and the standard error of its mean [e.g., Onyango and Plews, 1987, p. 249], suggests that the Arctic ice edge exhibits ∼4-15 spatial degrees of freedom to vary between years, depending on the model and the time of the year (Figure 2, right).However, if these independent fluctuations are superimposed by slower large-scale modes of variability (as revealed below), the relation in equation ( 8) gives only a lower bound for the DOF (see supporting information Text S1 and Figure S2).
The seasonal cycle of the fractional AEE is less coherent between the models, except for a tendency toward higher values in early autumn (Figure 2, right).A possible explanation is that during this time of the year, the ice edge is contiguous in the Arctic Ocean rather than made up of separate segments in the Pacific, the Labrador Sea, the Fram Strait, and the Barents Sea, resulting in fewer degrees of freedom of ice-edge variations.
Our estimate for the DOF of Arctic sea ice edge variability (≳4-15) is similar to the estimate for the DOF of Arctic sea ice thickness variability by Blanchard-Wrigglesworth and Bitz [2014] (3-14).The relation between these two measures is, however, not straightforward, one reason being that the former should scale with the perimeter whereas the latter should scale with the area of the sea ice cover (assuming constant spatial correlation lengths).

Potential Predictability
To estimate the potential predictability of the Arctic sea ice edge from the idealized forecast ensembles, we compute the mean pairwise IIEE over all possible pairs of ensemble members for all ensembles: where K is the number of ensembles and M is the ensemble size (compare Table 1).This implies that sequentially, each ensemble member is interpreted as truth, which is consistent with the so-called perfect-model approach [Collins, 2002].Note that IIEE pred is a function of forecast lead time.Division by IIEE clim (equation ( 7)) gives the normalized IIEE (IIEE norm ): 0 implies perfect predictability and 1 implies the complete loss of predictability.The same procedure is applied to derive normalized AEE and ME components.
In all models the Arctic ice-edge position remains substantially predictable within the first 6 months: the predictable fraction (1−IIEE norm ) decreases to ∼10-30% from 1 July to the end of the year (Figure 3, black curves).Subsequently, the predictable fraction slowly diminishes further and reaches almost zero (≤5%) after 3 years.The only exception is GFDL-CM3 where a predictable fraction of ∼10-15% still remains, implying that slow (up to decadal) large-scale modes of Arctic sea ice variability are relatively strong compared to faster small-scale fluctuations in GFDL-CM3 (see also supporting information Figure S1).
The general growth of IIEE norm is overlaid with a coherent seasonal cycle in AWI-CM, HadGEM1.2, and (to a lesser extent) GFDL-CM3: the predictable fraction is higher compared to the rest of the year in late winter and early spring, when the sea ice extent is maximum.This indicates that in these models the Arctic sea ice edge is subjected more strongly to slow modes of climate variability during this time of the year.
In AWI-CM, where the seasonality of IIEE norm is most pronounced, this can be linked to a strong mode of decadal variability in the northern North Atlantic where periods of active deep water formation in the Labrador Sea are interrupted by periods with virtually no deep water formation, accompanied by extensive late-winter/early-spring sea ice [Sidorenko et al., 2014].While this Atlantic Multidecadal Oscillation (AMO)-like variability mode affects the ocean surface throughout the year, its influence on the ice edge is largely constrained to those months when the ice edge reaches into the affected regions of the North Atlantic [see also Day et al., 2015, Figure 3].This explanation for seasonal reemergence of ice-edge predictability is closely related to the reemergence of memory in pan-Arctic sea ice extent from the melt season to the growth season reported in Blanchard-Wrigglesworth et al. [2011b], and consistent with the findings of Koenigk et al. [2012] and Day et al. [2012].

Geophysical Research Letters
10.1002/2015GL067232 Turning to the components that constitute the IIEE, first, it is important to note that AEE norm closely resembles the normalized root mean square error of sea ice extent reported in Day et al. [2015, Figure 6a].This corroborates that despite the subtle differences in the computation (mean absolute versus root mean square deviation; different concepts regarding climatology and drift removal), these two error metrics can be interpreted basically in the same way.
The misplacement error loses predictability faster than the absolute extent error in all models (Figure 3, compare blue and red curves).In fact, any predictability of the Arctic ice-edge position beyond 6 months lead time is largely attributable to the absolute extent error.Given that the climatological AEE contributes only ∼25-50% to the climatological IIEE in the first place (section 4), the slower relative error growth of AEE norm compared to ME norm (and thus IIEE norm ) implies that the fractional contribution of AEE pred to IIEE pred is even lower.While generally considerable intermodel differences exist, in September (2-3 months into the forecasts) the predictable fraction associated with the integrated ice-edge error (on average ∼50%) is consistently lower than the predictable fraction of just the absolute extent error (on average ∼70%).Accordingly, the AEE contributes on average only ∼25% to the IIEE in the first forecasted September-substantially less than the ∼40% contribution of the AEE to climatological September ice-edge variations (grey curve in Figure 2, right).This highlights that simple sea ice extent predictions capture only a small part, in this case one quarter, of the user-relevant information on the ice-edge position.
The difference in relative error growth rate raises the question whether a trivial (statistical) explanation exists.
We argue that the effect would be trivial if it occurred also in the above mentioned simplistic model of an ice edge composed of segments that are subject to independent identical random fluctuations.We have tested this by performing simple experiments based on autoregressive processes (using different orders), with each process resembling the extent anomaly of one ice-edge segment.When all segments fluctuate independently following the same statistics, we find indeed that the components of the IIEE grow at the same rate.However, if these independent fluctuations are superimposed by slower processes affecting multiple segments coherently, corresponding to larger-scale modes of variability, the AEE grows slower than the ME and, thus, the IIEE.
Our findings thus suggest that slower modes of Arctic ice-edge variability tend to be associated with larger spatial scales compared to faster fluctuations.For example, the slow modes of North Atlantic climate variability affect the whole Atlantic sector of the Arctic ice-edge coherently, contributing strongly to the absolute extent error.In contrast, shorter-lived atmospheric circulation anomalies may affect smaller segments of the ice edge, potentially causing an ice-edge advance in one region and a simultaneous retreat elsewhere, contributing mostly to the misplacement error.One may interpret this as scale-dependent predictability, as found in atmospheric forecasts [e.g., Boer, 2003].

Summary and Conclusions
The integrated ice-edge error (IIEE) introduced here, defined as the integral of all areas where the forecast and the truth disagree on the ice concentration being above or below 15%, has a number of attractive properties for verification purposes: It is conceptually simple, straightforward to derive from model and observational data, applicable to evaluate reforecasts over the past ∼35 years, decomposable into meaningful components, and it summarizes information that is highly relevant to potential forecast users-the accuracy of ice-edge position forecasts.We suggest that the IIEE might serve as a common headline score, similar to the 500 hPa geopotential height anomaly correlation in weather forecasting, to evaluate and compare sea ice forecasting systems.It should be noted that for real-world applications, forecast model data need to be interpolated to the observational grid, and it might be necessary to bias-correct the sea ice concentration thresholds used to determine the ice edge from forecast models and observations.
The concept of the IIEE can be transferred to any other contour.For example, operators of ice-strengthened ships might be more interested in the skill to forecast the position of a certain ice thickness contour.For verification purposes, however, pan-Arctic ice thickness contours might be insufficiently constrained in the past and current observations.
Applying the IIEE and its components to control simulations and idealized forecast ensembles generated for the APPOSITE project, we find that the pan-Arctic sea ice extent contributes only ∼25-50% to climatological variations of the Arctic ice edge.Based on the assumption that the ice edge is composed of a number of 10.1002/2015GL067232 segments that are subject to independent identical random fluctuations, this fraction translates into ∼4-15 spatial degrees of freedom of climatological ice-edge variations.
In all models analyzed, the IIEE grows considerably faster than just the component associated with the pan-Arctic sea ice extent.We argue that the difference in error growth rates is due to scale-dependent predictability beyond the trivial spatial-averaging effect: Slower and thus more predictable modes of ice-edge variability tend to be associated with larger spatial scales.For forecasts initialized on 1 July, the difference is particularly strong in September: While the predictable fraction for sea ice extent is ∼70%, the predictable fraction for the IIEE is only ∼50% on average.Our results suggest that the position of the Arctic sea ice edge is generally less predictable than the pan-Arctic sea ice extent.The potential skill of end-user relevant sea ice forecasts might thus be lower than previously thought.

Figure 1 .
Figure1.The sea ice edges (15% ice concentration contours) for two members of an AWI-CM idealized forecast ensemble on 15 September (initialized on 1 July of the same arbitrary year).Interpreting the blue contour as forecast and the red contour as truth (observations), the IIEE is the sum of all light blue (ice extent overestimated; O) and light red (ice extent underestimated; U) areas, compare equations (1)-(3).The depicted land-sea distribution corresponds to the AWI-CM ocean grid.

Figure
Figure(left)  The climatological IIEE at lag l = 10 year (see equation (7)) as a function of the time of the year for all models.(right) The fractional contribution of the AEE to IIEE clim (black scale on the left) and the corresponding degrees of freedom (DOF; red scale on the right, see equation (8)).
. Recently, several global climate modeling groups have conducted idealized experiments following a common protocol developed for the Arctic Predictability and Prediction On Seasonal to Interannual Timescales (APPOSITE)